Challenge of processing large amounts of data

  • How to process is quickly?
  • So how do we go about making the problem map so that it can be distributed computation?
  • Distributed/Parrallel Programming is hard

Mapreduce addresses all the challengs

  • Google's computational/data manipulation model
  • Elegant way to work with big data

Map Reduce and the New Software Stack

  • Covered in Chapter 2 in the Ullman book
  • Covered in Chapter 24 of the Joel Grus book
  • Python support packages https://pypi.python.org/pypi/mrjob
  • map() and reduce() are basic built in functions in python https://docs.python.org/2/library/functions.html

Built-in Python Functional Programming Tools

https://docs.python.org/2/tutorial/datastructures.html

Python Map Function

For each value in a sequence, process each one and output a new result for each element.


In [1]:
def cube(x): return x*x*x

map(cube,range(1,11))


Out[1]:
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]

If there are more parameters then each could be an array, and they are applied together one element at a time


In [7]:
seq = range(8)
def add(x,y): return x+y
map(add, seq,seq)


Out[7]:
[0, 2, 4, 6, 8, 10, 12, 14]

Python Reduce Function

For an array passed in, compute a single result. So a reduce function always has two paramters to carry the result foward with the next element in the sequence. The operation starts by using the first two values then passes the result with the next element to the function...


In [8]:
result = map(add, seq,seq)
reduce(add, result)  # adding each element of the result together


Out[8]:
56

If there is only one value in a sequence then that element is returned; if the sequence is empty, an exception is raised.

Word Count using python in a map reduce manner

Using the Canterbury Corpus Test file from http://compression.ca/act/files/canterbury.zip, We will attempt to count the each unique word.


In [9]:
import re
import pandas as pd
import numpy as np
aliceFile = open('data/canterbury/alice29.txt','r')
map1=[]

WORD_RE = re.compile(r"[\w']+")

# Create the map of words with prelminary counts
for line in aliceFile:
    for w in WORD_RE.findall(line):
        map1.append([w,1])

#sort the map            
map2 = sorted(map1)

#Separate the map into groups by the key values
df = pd.DataFrame(map2)

uniquewords = df[0].unique()
DataFrameDict = {elem : pd.DataFrame for elem in uniquewords}

for key in DataFrameDict.keys():
    DataFrameDict[key] = df[:][df[0] == key]

In [10]:
def wordcount(x,y):
    x[1] = x[1] + y[1]
    return x

#Add up the counts using reduce

for uw in uniquewords:
    uarray = np.array(DataFrameDict[uw])
    print reduce(wordcount,uarray)


["'" 1091L]
["'TIS" 1L]
["'Tis" 2L]
["'em" 3L]
["'tis" 2L]
['2' 1L]
['9' 1L]
['A' 17L]
['ADVENTURES' 1L]
["ALICE'S" 3L]
['ALL' 4L]
['AND' 3L]
['ARE' 6L]
['AT' 1L]
['Ada' 1L]
['Adventures' 2L]
['Advice' 1L]
['After' 6L]
['Ah' 5L]
['Ahem' 1L]
['Alas' 1L]
['Alice' 386L]
["Alice's" 9L]
['All' 5L]
['Allow' 1L]
['Always' 1L]
['Ambition' 1L]
['An' 5L]
['And' 67L]
['Ann' 4L]
['Antipathies' 1L]
['Anything' 1L]
['Are' 4L]
['Arithmetic' 1L]
['As' 17L]
['At' 9L]
['Atheling' 1L]
['Australia' 1L]
['BE' 1L]
['BEE' 1L]
['BEFORE' 1L]
['BEG' 1L]
['BEST' 2L]
['BOOTS' 1L]
['BUSY' 1L]
['Back' 1L]
['Be' 2L]
['Beau' 4L]
['Beautiful' 5L]
['Because' 1L]
['Before' 1L]
['Begin' 1L]
['Behead' 1L]
['Besides' 1L]
['Between' 1L]
['Bill' 12L]
["Bill's" 4L]
['Birds' 1L]
['Boots' 1L]
['Brandy' 1L]
['Bring' 1L]
['But' 37L]
['By' 4L]
['C' 1L]
['CAN' 4L]
['CHAPTER' 12L]
['CHORUS' 2L]
['COULD' 4L]
['COURT' 1L]
['CURTSEYING' 1L]
['Call' 4L]
['Can' 1L]
["Can't" 1L]
['Canary' 1L]
['Canterbury' 1L]
['Carroll' 1L]
['Cat' 24L]
["Cat's" 2L]
['Catch' 1L]
['Caterpillar' 26L]
["Caterpillar's" 1L]
['Caucus' 3L]
['Certainly' 1L]
['Cheshire' 7L]
['Chorus' 1L]
['Christmas' 1L]
['Classics' 1L]
['Coils' 1L]
['Collar' 1L]
['Come' 21L]
['Coming' 1L]
['Conqueror' 2L]
['Consider' 3L]
['Crab' 2L]
['Croquet' 1L]
['Curiouser' 1L]
['D' 1L]
['DOES' 1L]
["DON'T" 1L]
['DOTH' 1L]
['DRINK' 2L]
['Dear' 1L]
['Derision' 1L]
['Did' 3L]
['Digging' 2L]
['Dinah' 11L]
["Dinah'll" 2L]
["Dinah's" 1L]
['Dinn' 1L]
['Distraction' 1L]
['Do' 13L]
['Dodo' 13L]
['Does' 2L]
["Don't" 9L]
['Dormouse' 39L]
["Dormouse's" 1L]
['Down' 3L]
['Drawling' 3L]
['Drink' 1L]
['Drive' 1L]
['Duchess' 38L]
["Duchess's" 3L]
['Duck' 3L]
['Dutchess' 1L]
['EAT' 1L]
['EDITION' 1L]
['END' 1L]
['ESQ' 1L]
['EVEN' 1L]
['EVER' 1L]
['EVERYBODY' 1L]
['Each' 1L]
['Eaglet' 3L]
['Edgar' 1L]
['Edwin' 2L]
['Either' 1L]
['Elsie' 1L]
['England' 1L]
['English' 6L]
['Even' 1L]
['Everybody' 2L]
['Everything' 1L]
["Everything's" 1L]
['Evidence' 1L]
['Exactly' 3L]
['Explain' 2L]
['FATHER' 2L]
['FENDER' 1L]
['FIT' 2L]
['FOOT' 1L]
['FROM' 1L]
['FUL' 1L]
['FULCRUM' 1L]
['Fainting' 1L]
['Father' 2L]
['Fetch' 1L]
['Fifteenth' 1L]
['First' 7L]
['Fish' 2L]
['Five' 7L]
['Footman' 10L]
["Footman's" 1L]
['For' 13L]
['Forty' 1L]
['Found' 2L]
['Fourteenth' 1L]
['France' 1L]
['French' 4L]
['Frog' 1L]
['From' 1L]
['Fury' 2L]
['GAVE' 2L]
['Game' 1L]
['Geography' 1L]
['Get' 2L]
['Give' 3L]
['Go' 3L]
['Good' 1L]
['Grammar' 1L]
['Grief' 1L]
['Ground' 1L]
['Gryphon' 54L]
['HAD' 1L]
['HATED' 1L]
['HAVE' 4L]
['HE' 4L]
['HEARTHRUG' 1L]
['HEARTS' 1L]
['HER' 2L]
['HERE' 1L]
['HIGH' 1L]
['HIM' 3L]
['HIS' 1L]
['HOW' 1L]
["Hadn't" 1L]
['Half' 1L]
['Hand' 1L]
['Hardly' 1L]
['Hare' 31L]
['Has' 1L]
['Hatter' 54L]
["Hatter's" 1L]
['Have' 3L]
['He' 20L]
["He's" 1L]
['Heads' 1L]
['Hearts' 6L]
['Her' 2L]
['Herald' 1L]
['Here' 11L]
['Him' 1L]
['His' 1L]
['Hjckrrh' 1L]
['Hm' 1L]
['Hold' 4L]
['Hole' 1L]
['How' 27L]
['However' 13L]
['Hush' 2L]
['I' 408L]
["I'LL" 1L]
["I'M" 2L]
["I'VE" 1L]
["I'd" 11L]
["I'll" 30L]
["I'm" 57L]
["I've" 33L]
['IF' 2L]
['II' 1L]
['III' 1L]
['IN' 2L]
['INSIDE' 1L]
['IS' 8L]
['IT' 4L]
['ITS' 1L]
['IV' 1L]
['IX' 1L]
['Idiot' 1L]
['If' 16L]
['Imagine' 1L]
['Improve' 1L]
['In' 11L]
['Indeed' 1L]
['Involved' 1L]
['Is' 3L]
['It' 41L]
["It'll" 1L]
["It's" 22L]
['Jack' 1L]
['Just' 7L]
['KING' 1L]
['KNOW' 1L]
['Keep' 2L]
['King' 59L]
["King's" 2L]
['Kings' 1L]
['Knave' 9L]
['LEAVE' 1L]
['LESS' 1L]
['LITTLE' 2L]
['LOVE' 1L]
['Lacie' 1L]
['Last' 1L]
['Lastly' 1L]
['Latin' 1L]
['Latitude' 2L]
['Laughing' 1L]
['Leave' 1L]
['Let' 10L]
["Let's" 1L]
['Lewis' 1L]
['Like' 1L]
['Little' 1L]
['Lizard' 5L]
["Lizard's" 1L]
['Lobster' 5L]
['London' 1L]
['Long' 1L]
['Longitude' 2L]
['Look' 1L]
['Lory' 7L]
['Luckily' 1L]
['M' 3L]
["MARMALADE'" 1L]
['ME' 5L]
["ME'" 2L]
['MILE' 1L]
['MILLENNIUM' 1L]
['MINE' 2L]
['MORE' 2L]
['MUST' 3L]
['MYSELF' 1L]
['Ma' 1L]
["Ma'am" 1L]
['Mabel' 4L]
['Mad' 1L]
['Magpie' 1L]
['Majesty' 12L]
['March' 34L]
['Mary' 4L]
['May' 2L]
['Maybe' 1L]
['Mercia' 2L]
['Mind' 2L]
['Mine' 1L]
['Miss' 3L]
['Mock' 56L]
['Morcar' 2L]
['Mouse' 29L]
["Mouse's" 1L]
['Multiplication' 1L]
['My' 3L]
['Mystery' 2L]
['NEAR' 1L]
['NEVER' 2L]
['NO' 1L]
['NOT' 7L]
['Nay' 1L]
['Nearly' 1L]
['Never' 4L]
['New' 1L]
['Next' 2L]
['Nile' 1L]
['No' 20L]
['Nobody' 3L]
['Nonsense' 1L]
['Nor' 1L]
['Normans' 1L]
['Northumbria' 2L]
['Not' 9L]
['Nothing' 4L]
['Now' 12L]
['Number' 1L]
['O' 3L]
['OF' 3L]
['OLD' 2L]
['ONE' 4L]
['ORANGE' 1L]
['OURS' 1L]
['OUT' 1L]
['OUTSIDE' 1L]
['Of' 11L]
['Off' 11L]
['Oh' 35L]
['On' 4L]
['Once' 3L]
['One' 6L]
['Only' 3L]
['Or' 1L]
['Ou' 1L]
['Our' 1L]
['Owl' 3L]
['PERSONS' 1L]
['PLEASE' 1L]
['PLENTY' 1L]
['POCKET' 1L]
['PRECIOUS' 1L]
['PROVES' 1L]
['Panther' 3L]
['Paris' 2L]
['Party' 1L]
['Pat' 3L]
['Pennyworth' 1L]
['Pepper' 2L]
['Perhaps' 3L]
['Pig' 2L]
['Pigeon' 11L]
['Pinch' 1L]
['Please' 5L]
['Pool' 1L]
['Poor' 2L]
['Pray' 3L]
['Presently' 2L]
['Prizes' 2L]
['Puss' 1L]
['QUEEN' 1L]
['QUITE' 2L]
['Quadrille' 4L]
['Queen' 67L]
["Queen's" 7L]
['Queens' 1L]
['Quick' 1L]
["RABBIT'" 1L]
['RED' 1L]
['RETURNED' 1L]
['RIGHT' 1L]
['Rabbit' 41L]
["Rabbit's" 4L]
['Race' 1L]
['Read' 1L]
['Really' 2L]
['Reeling' 1L]
['Repeat' 1L]
['Right' 1L]
['Rome' 2L]
['Rule' 1L]
['Run' 1L]
['SAID' 2L]
['SHE' 4L]
["SHE'S" 1L]
['SHOES' 1L]
['SIT' 1L]
['SLUGGARD' 1L]
['SOME' 1L]
['SOMEBODY' 1L]
['SOMETHING' 1L]
['SOMEWHERE' 1L]
['SOUP' 1L]
['SWIM' 1L]
['Said' 4L]
['Same' 1L]
['Seals' 1L]
['Seaography' 1L]
['See' 1L]
['Sends' 1L]
['Sentence' 1L]
['Serpent' 3L]
['Seven' 5L]
['Sh' 1L]
['Shakespeare' 1L]
['Shall' 2L]
["Shan't" 1L]
['Shark' 1L]
['She' 35L]
["She'd" 1L]
["She'll" 1L]
["She's" 2L]
['Shy' 1L]
['Silence' 3L]
['Sing' 1L]
['Sir' 1L]
['Sit' 1L]
['Sixteenth' 1L]
['So' 27L]
['Soles' 1L]
['Some' 2L]
['Somebody' 1L]
['Soo' 7L]
['Soon' 1L]
['Sounds' 1L]
['Soup' 13L]
['Speak' 2L]
['Stand' 1L]
['Stigand' 1L]
['Still' 1L]
['Stole' 1L]
['Stolen' 1L]
['Stop' 1L]
['Story' 1L]
['Stretching' 1L]
['Stuff' 1L]
['Stupid' 1L]
['Such' 1L]
['Suddenly' 1L]
['Suppose' 2L]
['Suppress' 1L]
['Sure' 4L]
['Swim' 1L]
['T' 1L]
['THAN' 1L]
['THAT' 10L]
["THAT'S" 3L]
['THE' 9L]
['THEIR' 1L]
['THEN' 1L]
['THERE' 1L]
['THESE' 1L]
['THEY' 2L]
['THINK' 3L]
['THIS' 4L]
['THROUGH' 1L]
['TO' 3L]
['TOOK' 1L]
['TRUE' 1L]
['TWO' 2L]
['Table' 1L]
['Take' 4L]
['Tale' 1L]
['Talking' 1L]
['Tarts' 1L]
['Tea' 1L]
['Tears' 1L]
['Tell' 5L]
['Ten' 1L]
['Thank' 3L]
['That' 10L]
["That'll" 1L]
["That's" 16L]
['The' 108L]
['Their' 1L]
['Then' 21L]
['There' 25L]
["There's" 8L]
['Therefore' 1L]
['These' 2L]
['They' 18L]
["They're" 3L]
['Thinking' 1L]
['This' 17L]
['Those' 1L]
['Though' 1L]
['Tillie' 1L]
['Time' 3L]
['To' 1L]
['Too' 1L]
['Tortoise' 3L]
['Treacle' 2L]
['Trims' 1L]
['Turn' 3L]
['Turtle' 57L]
["Turtle's" 2L]
['Tut' 1L]
['Twenty' 1L]
['Twinkle' 3L]
['Two' 6L]
['UNimportant' 2L]
['Ugh' 2L]
['Uglification' 2L]
['Up' 2L]
['V' 1L]
['VERY' 13L]
['VI' 1L]
['VII' 1L]
['VIII' 1L]
['VOICE' 1L]
['Very' 5L]
['Visit' 1L]
['W' 1L]
['WAISTCOAT' 1L]
['WAS' 4L]
['WASHING' 1L]
['WATCH' 1L]
['WE' 1L]
['WHAT' 6L]
['WHATEVER' 1L]
['WILL' 3L]
['WILLIAM' 2L]
['WITH' 1L]
['WONDERLAND' 1L]
['WOULD' 5L]
['Waiting' 1L]
['Wake' 2L]
['Was' 1L]
['We' 10L]
['Well' 23L]
['What' 40L]
["What's" 2L]
['When' 10L]
['Where' 4L]
["Where's" 1L]
['Which' 8L]
['While' 5L]
['White' 22L]
['Who' 13L]
["Who's" 2L]
['Whoever' 1L]
['Why' 27L]
['Will' 6L]
['William' 5L]
["William's" 1L]
['With' 4L]
['Wonderland' 2L]
['Would' 8L]
["Wouldn't" 1L]
['Wow' 2L]
['Write' 1L]
['Writhing' 1L]
['X' 1L]
['XI' 1L]
['XII' 1L]
['YET' 1L]
['YOU' 14L]
["YOU'D" 1L]
['YOUR' 7L]
['YOURS' 1L]
['Yes' 12L]
['Yet' 3L]
['You' 39L]
["You'd" 1L]
["You'll" 2L]
["You're" 8L]
["You've" 1L]
['Your' 2L]
['Zealand' 1L]
['_I_' 2L]
['a' 615L]
['abide' 1L]
['able' 1L]
['about' 94L]
['above' 3L]
['absence' 1L]
['absurd' 2L]
['acceptance' 1L]
['accident' 2L]
['accidentally' 1L]
['account' 1L]
['accounting' 1L]
['accounts' 1L]
['accusation' 1L]
['accustomed' 1L]
['ache' 1L]
['across' 5L]
['act' 1L]
['actually' 1L]
['added' 23L]
['adding' 1L]
['addressed' 2L]
['addressing' 1L]
['adjourn' 1L]
['adoption' 1L]
['advance' 3L]
['advantage' 3L]
['adventures' 4L]
['advice' 1L]
['advisable' 2L]
['advise' 1L]
['affair' 1L]
['affectionately' 1L]
['afford' 1L]
['afore' 1L]
['afraid' 12L]
['after' 37L]
['afterwards' 2L]
['again' 82L]
['against' 9L]
['age' 4L]
['ago' 2L]
['agony' 1L]
['agree' 2L]
['air' 15L]
['airs' 1L]
['alarm' 2L]
['alarmed' 1L]
['alas' 3L]
['alive' 3L]
['all' 173L]
['allow' 2L]
['almost' 6L]
['alone' 4L]
['along' 6L]
['aloud' 5L]
['already' 2L]
['also' 2L]
['altered' 1L]
['alternately' 1L]
['altogether' 5L]
['always' 12L]
['am' 15L]
['among' 12L]
['an' 52L]
['ancient' 1L]
['and' 802L]
['anger' 2L]
['angrily' 9L]
['angry' 5L]
['animal' 1L]
["animal's" 1L]
['animals' 4L]
['annoy' 1L]
['annoyed' 1L]
['another' 22L]
['answer' 9L]
['answered' 4L]
['answers' 1L]
['anxious' 3L]
['anxiously' 14L]
['any' 39L]
['anything' 19L]
['anywhere' 1L]
['appealed' 1L]
['appear' 2L]
['appearance' 1L]
['appeared' 8L]
['appearing' 1L]
['applause' 1L]
['apple' 1L]
['apples' 2L]
['arch' 1L]
['archbishop' 2L]
['arches' 4L]
['are' 44L]
['argue' 1L]
['argued' 1L]
['argument' 4L]
['arguments' 1L]
['arm' 15L]
['arms' 6L]
['around' 3L]
['arranged' 1L]
['arrived' 1L]
['arrow' 1L]
['arrum' 1L]
['as' 246L]
['ashamed' 2L]
['ask' 11L]
['askance' 1L]
['asked' 17L]
['asking' 5L]
['asleep' 8L]
['assembled' 2L]
['at' 201L]
['ate' 1L]
['atom' 2L]
['attempt' 1L]
['attempted' 1L]
['attempts' 1L]
['attended' 1L]
['attending' 3L]
['attends' 1L]
['audibly' 1L]
['authority' 2L]
['avoid' 1L]
['away' 25L]
['awfully' 1L]
['axes' 1L]
['axis' 1L]
['baby' 14L]
['back' 38L]
['backs' 1L]
['bad' 2L]
['bag' 1L]
['baked' 1L]
['balanced' 1L]
['balls' 1L]
['bank' 3L]
['banks' 1L]
['banquet' 1L]
['bark' 2L]
['barking' 1L]
['barley' 1L]
['barrowful' 2L]
['bat' 3L]
['bathing' 1L]
['bats' 4L]
['bawled' 1L]
['be' 145L]
['beak' 1L]
['bear' 2L]
['beast' 1L]
['beasts' 2L]
['beat' 4L]
['beating' 2L]
['beauti' 1L]
['beautiful' 8L]
['beautifully' 2L]
['beautify' 1L]
['became' 2L]
['because' 14L]
['become' 5L]
['becoming' 1L]
['bed' 1L]
['beds' 2L]
['been' 38L]
['before' 36L]
['beg' 7L]
['began' 58L]
['begged' 1L]
['begin' 12L]
['beginning' 14L]
['begins' 4L]
['begun' 7L]
['beheaded' 3L]
['beheading' 1L]
['behind' 13L]
['being' 19L]
['believe' 9L]
['believed' 1L]
['bells' 1L]
['belong' 1L]
['belongs' 2L]
['beloved' 1L]
['below' 3L]
['belt' 1L]
['bend' 2L]
['bent' 1L]
['besides' 3L]
['best' 10L]
['better' 13L]
["better'" 1L]
['between' 5L]
['bill' 1L]
['bird' 2L]
['birds' 9L]
['birthday' 1L]
['bit' 16L]
['bite' 2L]
['bitter' 1L]
['blacking' 1L]
['blades' 1L]
['blame' 1L]
['blasts' 2L]
['bleeds' 1L]
['blew' 2L]
['blow' 2L]
['blown' 1L]
['blows' 1L]
['body' 2L]
['boldly' 1L]
['bone' 1L]
['bones' 1L]
['book' 11L]
['books' 2L]
['boon' 1L]
['boots' 2L]
['bore' 1L]
['both' 14L]
['bother' 1L]
['bottle' 10L]
['bottom' 4L]
['bough' 1L]
['bound' 1L]
['bowed' 4L]
['bowing' 1L]
['box' 10L]
['boxed' 1L]
['boy' 3L]
['brain' 1L]
['branch' 1L]
['branches' 2L]
['brass' 1L]
['brave' 1L]
['bread' 7L]
['break' 2L]
['breath' 4L]
['breathe' 3L]
['breeze' 1L]
['bright' 8L]
['brightened' 2L]
['bring' 2L]
['bringing' 3L]
['bristling' 1L]
['broke' 2L]
['broken' 6L]
["brother's" 1L]
['brought' 3L]
['brown' 2L]
['brush' 1L]
['brushing' 1L]
['burn' 2L]
['burning' 1L]
['burnt' 1L]
['burst' 1L]
['bursting' 1L]
['busily' 4L]
['business' 7L]
['busy' 1L]
['but' 133L]
['butter' 9L]
['buttercup' 1L]
['buttered' 1L]
['butterfly' 1L]
['buttons' 1L]
['by' 54L]
['bye' 2L]
['cackled' 1L]
['cake' 3L]
['cakes' 3L]
['call' 5L]
['called' 15L]
['calling' 1L]
['calmly' 1L]
['came' 40L]
['camomile' 1L]
['can' 30L]
["can't" 27L]
['candle' 3L]
['cannot' 1L]
['canvas' 1L]
['capering' 1L]
['capital' 4L]
['cardboard' 1L]
['cards' 3L]
['care' 4L]
['carefully' 3L]
['cares' 2L]
['carried' 4L]
['carrier' 1L]
['carry' 1L]
['carrying' 2L]
['cart' 1L]
['cartwheels' 1L]
['case' 5L]
['cat' 11L]
['catch' 3L]
['catching' 2L]
['caterpillar' 1L]
['cats' 13L]
['cattle' 1L]
['caught' 3L]
['cauldron' 2L]
['cause' 3L]
['caused' 2L]
['cautiously' 3L]
['ceiling' 1L]
['centre' 1L]
['certain' 3L]
['certainly' 13L]
['chain' 1L]
['chains' 1L]
['chair' 1L]
['chance' 4L]
['chanced' 1L]
['change' 13L]
['changed' 8L]
['changes' 2L]
['changing' 2L]
['character' 1L]
['charges' 1L]
['chatte' 1L]
['cheap' 1L]
['cheated' 1L]
['checked' 3L]
['cheeks' 1L]
['cheered' 3L]
['cheerfully' 1L]
['cherry' 1L]
['chief' 1L]
['child' 11L]
['childhood' 1L]
['children' 10L]
['chimney' 6L]
['chimneys' 1L]
['chin' 7L]
['choice' 2L]
['choke' 1L]
['choked' 3L]
['choking' 1L]
['choosing' 1L]
['chop' 1L]
['chorus' 3L]
['chose' 2L]
['chrysalis' 1L]
['chuckled' 1L]
['circle' 1L]
['circumstances' 1L]
['civil' 3L]
['clamour' 1L]
['clapping' 1L]
['clasped' 1L]
['claws' 2L]
['clean' 1L]
['clear' 2L]
['cleared' 1L]
['clearer' 1L]
['clearly' 1L]
['clever' 2L]
['climb' 1L]
['clinging' 1L]
['clock' 2L]
['close' 13L]
['closed' 2L]
['closely' 1L]
['closer' 1L]
['clubs' 1L]
['coast' 1L]
['coaxing' 2L]
['cold' 1L]
['collected' 2L]
['come' 25L]
['comes' 2L]
['comfits' 2L]
['comfort' 1L]
['comfortable' 1L]
['comfortably' 1L]
['coming' 8L]
['common' 1L]
['commotion' 1L]
['company' 1L]
['complained' 1L]
['complaining' 1L]
['completely' 1L]
['concert' 2L]
['concluded' 2L]
['conclusion' 2L]
['condemn' 1L]
['conduct' 1L]
['confused' 4L]
['confusing' 3L]
['confusion' 5L]
['conger' 1L]
['conquest' 1L]
['consented' 1L]
['consider' 1L]
['considered' 3L]
['considering' 3L]
['constant' 2L]
['consultation' 1L]
['contempt' 1L]
['contemptuous' 1L]
['contemptuously' 2L]
['content' 1L]
['continued' 9L]
['contradicted' 1L]
['conversation' 10L]
['conversations' 1L]
['cook' 13L]
['cool' 2L]
['corner' 4L]
['corners' 1L]
['cost' 1L]
['could' 73L]
["couldn't" 9L]
['counting' 1L]
['country' 1L]
['couple' 1L]
['couples' 1L]
['courage' 3L]
['course' 26L]
['court' 17L]
['courtiers' 2L]
['coward' 1L]
['crab' 1L]
['crash' 3L]
['crashed' 1L]
['crawled' 1L]
['crawling' 1L]
['crazy' 1L]
['creature' 4L]
['creatures' 10L]
['creep' 1L]
['crept' 1L]
['cried' 20L]
['cries' 1L]
['crimson' 2L]
['crocodile' 1L]
['croquet' 8L]
['croqueted' 1L]
['croqueting' 1L]
['cross' 3L]
['crossed' 3L]
['crossly' 1L]
['crouched' 1L]
['crowd' 4L]
['crowded' 5L]
['crown' 3L]
['crumbs' 4L]
['crust' 1L]
['cry' 3L]
['crying' 2L]
['cucumber' 2L]
['cunning' 1L]
['cup' 2L]
['cupboards' 2L]
['cur' 1L]
['curiosity' 5L]
['curious' 19L]
['curiouser' 1L]
['curled' 2L]
['curls' 1L]
['curly' 1L]
['currants' 1L]
['curtain' 1L]
['curtsey' 1L]
['curving' 1L]
['cushion' 2L]
['custard' 1L]
['custody' 2L]
['cut' 5L]
['cutting' 1L]
['dainties' 1L]
['daisies' 1L]
['daisy' 1L]
['dance' 13L]
['dancing' 2L]
['dare' 5L]
['daresay' 1L]
['dark' 3L]
['darkness' 1L]
['dates' 1L]
['daughter' 1L]
['day' 29L]
['days' 4L]
['dead' 4L]
['deal' 12L]
['dear' 28L]
['dears' 3L]
['death' 1L]
['decided' 3L]
['decidedly' 4L]
['declare' 2L]
['declared' 1L]
['deep' 7L]
['deepest' 1L]
['deeply' 4L]
['delay' 1L]
['delight' 3L]
['delighted' 2L]
['delightful' 2L]
['denial' 1L]
['denied' 2L]
['denies' 1L]
['deny' 2L]
['denying' 1L]
['depends' 1L]
['deserved' 1L]
['desk' 1L]
['desks' 1L]
['despair' 1L]
['desperate' 1L]
['desperately' 1L]
['diamonds' 1L]
['did' 60L]
["didn't" 14L]
['die' 1L]
['died' 1L]
['different' 9L]
['difficult' 2L]
['difficulties' 1L]
['difficulty' 4L]
['dig' 1L]
['digging' 2L]
['diligently' 1L]
['dinn' 1L]
['dinner' 2L]
['dipped' 2L]
['directed' 2L]
['direction' 5L]
['directions' 3L]
['directly' 2L]
['disagree' 1L]
['disappeared' 2L]
['disappointment' 1L]
['disgust' 1L]
['dish' 4L]
['dishes' 2L]
['dismay' 1L]
['disobey' 1L]
['dispute' 2L]
['distance' 8L]
['distant' 2L]
['dive' 1L]
['do' 68L]
['dodged' 1L]
['does' 6L]
["doesn't" 16L]
['dog' 2L]
["dog's" 1L]
['dogs' 3L]
['doing' 5L]
["don't" 51L]
['done' 15L]
['door' 30L]
['doors' 2L]
['doorway' 1L]
['doth' 2L]
['double' 1L]
['doubled' 1L]
['doubling' 1L]
['doubt' 4L]
['doubtful' 2L]
['doubtfully' 2L]
['down' 99L]
['downward' 1L]
['downwards' 1L]
['doze' 1L]
['dozing' 1L]
['draggled' 1L]
['draw' 7L]
['drawing' 1L]
['dreadful' 2L]
['dreadfully' 6L]
['dream' 7L]
['dreamed' 1L]
['dreaming' 1L]
['dreamy' 1L]
['dressed' 1L]
['drew' 5L]
['dried' 1L]
['driest' 1L]
['drink' 4L]
['drinking' 1L]
['dripping' 1L]
['drive' 1L]
['drop' 1L]
['dropped' 5L]
['dropping' 1L]
['drowned' 1L]
['drunk' 2L]
['dry' 8L]
['duck' 1L]
['dull' 3L]
['dunce' 1L]
['e' 6L]
['each' 7L]
['eager' 3L]
['eagerly' 8L]
['ear' 6L]
['earls' 2L]
['earnestly' 2L]
['ears' 5L]
['earth' 4L]
['easily' 3L]
['easy' 2L]
['eat' 17L]
['eaten' 1L]
['eating' 1L]
['eats' 1L]
['edge' 3L]
['editions' 2L]
['educations' 1L]
['eel' 2L]
['eels' 1L]
['effect' 3L]
['egg' 1L]
['eggs' 5L]
['eh' 1L]
['either' 9L]
['elbow' 3L]
['elbows' 1L]
['elegant' 1L]
['eleventh' 1L]
['else' 11L]
["else's" 1L]
['emphasis' 1L]
['empty' 1L]
['encourage' 1L]
['encouraged' 1L]
['encouraging' 2L]
['end' 17L]
['ending' 2L]
['energetic' 1L]
['engaged' 1L]
['engine' 1L]
['engraved' 1L]
['enjoy' 1L]
['ennyworth' 1L]
['enormous' 1L]
['enough' 18L]
['entangled' 2L]
['entirely' 2L]
['entrance' 1L]
['escape' 4L]
['est' 1L]
['even' 17L]
['evening' 5L]
['ever' 19L]
['every' 13L]
['everybody' 5L]
['everything' 11L]
["everything's" 1L]
['evidence' 6L]
['evidently' 1L]
['exact' 1L]
['exactly' 5L]
['examine' 2L]
['examining' 1L]
['excellent' 2L]
['except' 4L]
['exclaimed' 6L]
['exclamation' 1L]
['execute' 1L]
['executed' 6L]
['executes' 1L]
['execution' 3L]
['executioner' 5L]
["executioner's" 1L]
['executions' 2L]
['existence' 1L]
['expected' 1L]
['expecting' 3L]
['experiment' 2L]
['explain' 8L]
['explained' 1L]
['explanation' 2L]
['explanations' 1L]
['expressing' 1L]
['expression' 1L]
['extra' 1L]
['extraordinary' 2L]
['extras' 1L]
['extremely' 2L]
['eye' 7L]
['eyed' 1L]
['eyelids' 1L]
['eyes' 29L]
['face' 15L]
['faces' 5L]
['fact' 8L]
['fading' 1L]
['failure' 1L]
['faint' 1L]
['faintly' 1L]
['fair' 1L]
['fairly' 1L]
['fairy' 1L]
['fall' 7L]
['fallen' 4L]
['falling' 2L]
['familiarly' 1L]
['family' 1L]
['fan' 10L]
['fancied' 2L]
['fancy' 7L]
['fancying' 1L]
['fanned' 1L]
['fanning' 1L]
['far' 13L]
['farm' 1L]
['farmer' 1L]
['farther' 1L]
['fashion' 2L]
['fast' 4L]
['faster' 3L]
['fat' 1L]
['father' 2L]
['favoured' 1L]
['favourite' 1L]
['fear' 4L]
['feared' 1L]
['feather' 1L]
['feathers' 1L]
['feeble' 2L]
['feebly' 1L]
['feel' 8L]
['feeling' 7L]
['feelings' 2L]
['feet' 19L]
['fell' 6L]
['fellow' 4L]
['fellows' 1L]
['felt' 23L]
['ferrets' 2L]
['fetch' 6L]
['few' 9L]
['fidgeted' 1L]
['field' 1L]
['fifteen' 1L]
['fifth' 1L]
['fig' 1L]
['fight' 2L]
['fighting' 1L]
['figure' 3L]
['figures' 1L]
['filled' 3L]
['fills' 1L]
['find' 21L]
['finding' 3L]
['finds' 1L]
['fine' 2L]
['finger' 5L]
['finish' 5L]
['finished' 12L]
['finishing' 1L]
['fire' 4L]
['fireplace' 1L]
['first' 44L]
['fish' 6L]
['fishes' 1L]
['fit' 1L]
['fits' 1L]
['fitted' 1L]
['five' 1L]
['fix' 1L]
['fixed' 1L]
['flame' 1L]
['flamingo' 5L]
['flamingoes' 2L]
['flapper' 1L]
['flappers' 1L]
['flashed' 1L]
['flat' 2L]
['flavour' 1L]
['flew' 1L]
['flinging' 1L]
['flock' 1L]
['floor' 3L]
['flower' 2L]
['flowers' 2L]
['flown' 1L]
['flung' 1L]
['flurry' 1L]
['flustered' 1L]
['fluttered' 1L]
['fly' 3L]
['flying' 1L]
['folded' 3L]
['folding' 1L]
['follow' 2L]
['followed' 8L]
['follows' 3L]
['fond' 4L]
['foolish' 1L]
['foot' 9L]
['footman' 3L]
['footmen' 1L]
['footsteps' 2L]
['for' 140L]
['forehead' 2L]
['forepaws' 1L]
['forget' 2L]
['forgetting' 3L]
['forgot' 2L]
['forgotten' 6L]
['fork' 1L]
['form' 1L]
['fortunately' 1L]
['forwards' 1L]
['found' 30L]
['fountains' 2L]
['four' 8L]
['fourth' 1L]
['frame' 1L]
['frames' 1L]
['free' 3L]
['friend' 3L]
['friends' 2L]
['fright' 2L]
['frighten' 1L]
['frightened' 7L]
['frog' 2L]
['from' 34L]
['front' 2L]
['frontispiece' 1L]
['frowning' 4L]
['frying' 1L]
['full' 6L]
['fumbled' 1L]
['fun' 3L]
['funny' 3L]
['fur' 3L]
['furious' 1L]
['furiously' 1L]
['furrow' 1L]
['furrows' 1L]
['further' 3L]
['fury' 1L]
['gained' 1L]
['gallons' 1L]
['game' 11L]
["game's" 1L]
['games' 1L]
['garden' 16L]
['gardeners' 8L]
['gather' 1L]
['gave' 13L]
['gay' 1L]
['gazing' 1L]
['general' 3L]
['generally' 7L]
['gently' 3L]
['get' 44L]
['getting' 22L]
['giddy' 2L]
['girl' 4L]
['girls' 3L]
['give' 9L]
['given' 1L]
['giving' 2L]
['glad' 11L]
['glanced' 1L]
['glaring' 1L]
['glass' 10L]
['globe' 1L]
['gloomily' 1L]
['gloves' 11L]
['go' 47L]
['goes' 7L]
['going' 27L]
['golden' 7L]
['goldfish' 2L]
['gone' 13L]
['good' 26L]
['goose' 2L]
['got' 45L]
['graceful' 1L]
['grand' 3L]
['grant' 1L]
['grass' 4L]
['grave' 3L]
['gravely' 3L]
['gravy' 1L]
['grazed' 1L]
['great' 39L]
['green' 4L]
['grew' 1L]
['grey' 1L]
['grin' 6L]
['grinned' 3L]
['grinning' 1L]
['grins' 1L]
['ground' 7L]
['grow' 13L]
['growing' 11L]
['growl' 3L]
['growled' 1L]
['growling' 1L]
['growls' 1L]
['grown' 7L]
['grumbled' 1L]
['grunt' 1L]
['grunted' 4L]
['guard' 1L]
['guess' 3L]
['guessed' 3L]
['guests' 3L]
['guilt' 1L]
['guinea' 6L]
['had' 177L]
["hadn't" 7L]
['hair' 7L]
['half' 22L]
['hall' 9L]
['hand' 20L]
['handed' 3L]
['hands' 12L]
['handsome' 1L]
['handwriting' 1L]
['hanging' 3L]
['happen' 8L]
['happened' 7L]
['happening' 1L]
['happens' 5L]
['happy' 1L]
['hard' 8L]
['hardly' 11L]
['harm' 1L]
['has' 6L]
["hasn't" 2L]
['haste' 1L]
['hastily' 16L]
['hat' 1L]
['hatching' 1L]
['hate' 2L]
['hatter' 1L]
['hatters' 1L]
['have' 73L]
["haven't" 8L]
['having' 10L]
['he' 98L]
["he'd" 1L]
["he'll" 1L]
["he's" 2L]
['head' 49L]
["head's" 1L]
['heads' 9L]
['heap' 1L]
['hear' 14L]
['heard' 30L]
['hearing' 4L]
['heart' 2L]
['hearth' 1L]
['hearts' 1L]
['heavy' 2L]
['hedge' 2L]
['hedgehog' 7L]
['hedgehogs' 3L]
['hedges' 1L]
['heels' 1L]
['height' 5L]
['held' 4L]
['help' 9L]
['helped' 1L]
['helpless' 1L]
['her' 243L]
['here' 39L]
['hers' 4L]
['herself' 83L]
['hid' 1L]
['hide' 1L]
['high' 15L]
['highest' 1L]
['him' 39L]
['himself' 6L]
['hint' 2L]
['hippopotamus' 1L]
['his' 94L]
['hiss' 1L]
['histories' 1L]
['history' 7L]
['hit' 2L]
['hoarse' 3L]
['hoarsely' 1L]
['hold' 6L]
['holding' 3L]
['hole' 4L]
['holiday' 1L]
['hollow' 1L]
['home' 5L]
['honest' 1L]
['honour' 4L]
['hookah' 5L]
['hope' 3L]
['hoped' 1L]
['hopeful' 1L]
['hopeless' 1L]
['hoping' 3L]
['horse' 1L]
['hot' 7L]
['hour' 2L]
['hours' 4L]
['house' 18L]
['housemaid' 1L]
['houses' 1L]
['how' 40L]
['however' 7L]
['howled' 1L]
['howling' 3L]
['humble' 1L]
['humbly' 2L]
['hundred' 1L]
['hung' 1L]
['hungry' 3L]
['hunting' 3L]
['hurried' 11L]
['hurriedly' 2L]
['hurry' 11L]
['hurrying' 1L]
['hurt' 3L]
['hush' 1L]
['idea' 15L]
['idiotic' 1L]
['if' 78L]
['ignorant' 1L]
['ill' 2L]
['imagine' 1L]
['imitated' 1L]
['immediate' 1L]
['immediately' 3L]
['immense' 1L]
['impatient' 1L]
['impatiently' 5L]
['impertinent' 1L]
['important' 7L]
['impossible' 3L]
['in' 356L]
['incessantly' 1L]
['inches' 6L]
['inclined' 1L]
['indeed' 15L]
['indignant' 1L]
['indignantly' 4L]
['injure' 1L]
['ink' 1L]
['inkstand' 1L]
['inquired' 1L]
['inquisitively' 1L]
['inside' 1L]
['insolence' 1L]
['instance' 3L]
['instantly' 5L]
['instead' 3L]
['insult' 1L]
['interest' 1L]
['interesting' 5L]
['interrupt' 1L]
['interrupted' 9L]
['interrupting' 2L]
['into' 67L]
['introduce' 2L]
['introduced' 1L]
['invent' 1L]
['invented' 1L]
['invitation' 2L]
['invited' 2L]
['inwards' 1L]
['irons' 1L]
['irritated' 1L]
['is' 97L]
["isn't" 7L]
['it' 485L]
["it'll" 7L]
["it's" 35L]
['its' 55L]
['itself' 14L]
['jar' 2L]
['jaw' 1L]
['jaws' 2L]
['jelly' 1L]
['jogged' 1L]
['join' 9L]
['joined' 3L]
['journey' 1L]
['joys' 1L]
['judge' 4L]
['judging' 1L]
['jug' 1L]
['jumped' 6L]
['jumping' 4L]
['juror' 1L]
['jurors' 4L]
['jury' 22L]
['jurymen' 4L]
['just' 45L]
['justice' 1L]
['keep' 9L]
['keeping' 2L]
['kept' 13L]
['kettle' 1L]
['key' 9L]
['kick' 3L]
['kid' 5L]
['kill' 1L]
['killing' 1L]
['kills' 1L]
['kind' 7L]
['kindly' 2L]
['king' 1L]
['kiss' 1L]
['kissed' 1L]
['kitchen' 4L]
['knee' 5L]
['kneel' 1L]
['knelt' 1L]
['knew' 14L]
['knife' 3L]
['knock' 1L]
['knocked' 1L]
['knocking' 3L]
['knot' 2L]
['know' 87L]
['knowing' 2L]
['knowledge' 3L]
['known' 1L]
['knows' 2L]
['knuckles' 1L]
['label' 2L]
['labelled' 1L]
['lad' 1L]
['ladder' 1L]
['lady' 3L]
['laid' 2L]
['lamps' 1L]
['land' 1L]
['languid' 1L]
['lap' 2L]
['large' 33L]
['larger' 7L]
['largest' 1L]
['lark' 1L]
['last' 32L]
['lasted' 2L]
['late' 6L]
['lately' 1L]
['later' 3L]
['laugh' 1L]
['laughed' 2L]
['laughing' 1L]
['laughter' 1L]
['law' 2L]
['lay' 4L]
['lazily' 1L]
['lazy' 1L]
['leaders' 1L]
['leading' 1L]
['leaning' 2L]
['leant' 1L]
['leap' 1L]
['learn' 7L]
['learned' 1L]
['learning' 2L]
['learnt' 2L]
['least' 9L]
['leave' 7L]
['leaves' 6L]
['leaving' 1L]
['led' 4L]
['ledge' 1L]
['left' 14L]
['lefthand' 2L]
['legged' 2L]
['legs' 3L]
['length' 1L]
['less' 3L]
['lessen' 1L]
['lesson' 3L]
['lessons' 10L]
['lest' 1L]
['let' 7L]
["let's" 4L]
['letter' 3L]
['letters' 1L]
['licking' 1L]
['lie' 2L]
['life' 12L]
['lifted' 1L]
['like' 84L]
['liked' 6L]
['likely' 5L]
['likes' 1L]
['limbs' 1L]
['line' 2L]
['lines' 1L]
['linked' 1L]
['lips' 1L]
['list' 3L]
['listen' 7L]
['listened' 1L]
['listeners' 1L]
['listening' 3L]
['lit' 1L]
['little' 125L]
['live' 8L]
['lived' 3L]
['livery' 3L]
['lives' 4L]
['living' 2L]
['lobster' 2L]
['lobsters' 6L]
['lock' 1L]
['locked' 1L]
['locks' 2L]
['lodging' 1L]
['lonely' 2L]
['long' 31L]
['longed' 2L]
['longer' 3L]
['look' 28L]
['looked' 45L]
['looking' 32L]
['loose' 1L]
['lose' 1L]
['losing' 1L]
['lost' 3L]
['loud' 6L]
['louder' 1L]
['loudly' 3L]
['love' 2L]
['loveliest' 1L]
['lovely' 2L]
['loving' 1L]
['low' 15L]
['lower' 1L]
['lowing' 1L]
['luckily' 1L]
['lullaby' 1L]
['lying' 8L]
['m' 1L]
['ma' 1L]
['machines' 1L]
['mad' 14L]
['made' 30L]
['magic' 1L]
['make' 27L]
['makes' 11L]
['making' 8L]
['mallets' 1L]
['man' 5L]
['manage' 7L]
['managed' 4L]
['managing' 1L]
['manner' 2L]
['manners' 1L]
['many' 12L]
['maps' 1L]
['marched' 1L]
['mark' 3L]
['marked' 6L]
['master' 4L]
['matter' 9L]
['matters' 2L]
['may' 11L]
['maybe' 1L]
["mayn't" 1L]
['me' 61L]
['meal' 1L]
['mean' 10L]
['meaning' 8L]
['means' 5L]
['meant' 5L]
['meanwhile' 1L]
['measure' 1L]
['meat' 1L]
['meekly' 2L]
['meet' 2L]
['meeting' 1L]
['melancholy' 6L]
['memorandum' 1L]
['memory' 1L]
["men'" 1L]
['mentioned' 3L]
['merely' 2L]
['merrily' 1L]
['messages' 2L]
['met' 3L]
['mice' 4L]
['middle' 7L]
['might' 28L]
['mile' 1L]
['miles' 3L]
['milk' 2L]
['mind' 9L]
['minded' 1L]
['minding' 1L]
['mine' 7L]
['mineral' 1L]
['minute' 21L]
['minutes' 11L]
['mischief' 1L]
['miserable' 2L]
['miss' 1L]
['missed' 2L]
['mistake' 3L]
['mixed' 2L]
['moderate' 1L]
['modern' 1L]
['moment' 29L]
["moment's" 2L]
['month' 2L]
['moon' 1L]
['moral' 8L]
['morals' 1L]
['more' 47L]
['morning' 5L]
['morsel' 1L]
['most' 8L]
['mostly' 2L]
['mournful' 1L]
['mournfully' 1L]
['mouse' 14L]
['mouth' 10L]
['mouths' 4L]
['move' 3L]
['moved' 5L]
['moving' 3L]
['much' 51L]
['muchness' 3L]
['muddle' 1L]
['murder' 1L]
['murdering' 1L]
['muscular' 1L]
['mushroom' 8L]
['music' 3L]
['must' 41L]
['mustard' 3L]
['muttered' 2L]
['muttering' 3L]
['my' 55L]
['myself' 6L]
['name' 10L]
['names' 2L]
['narrow' 2L]
['nasty' 1L]
['natural' 4L]
['natured' 1L]
['naturedly' 1L]
['near' 14L]
['nearer' 5L]
['nearly' 10L]
['neat' 1L]
['neatly' 2L]
['neck' 7L]
["needn't" 3L]
['needs' 1L]
['neighbour' 1L]
['neighbouring' 1L]
['neither' 2L]
['nervous' 5L]
['nest' 1L]
['never' 41L]
["never'" 1L]
['nevertheless' 1L]
['new' 4L]
['newspapers' 1L]
['next' 28L]
['nibbled' 2L]
['nibbling' 3L]
['nice' 6L]
['nicely' 2L]
['night' 5L]
['nine' 5L]
['no' 69L]
['nobody' 5L]
['nodded' 1L]
['noise' 3L]
['noises' 1L]
['none' 4L]
['nonsense' 6L]
['nor' 2L]
['nose' 7L]
["nose'" 1L]
['not' 128L]
["not'" 1L]
['note' 2L]
['nothing' 30L]
['notice' 5L]
['noticed' 8L]
['noticing' 1L]
['notion' 3L]
['now' 48L]
['nowhere' 2L]
['number' 4L]
['nurse' 3L]
['nursing' 3L]
["o'clock" 3L]
['obliged' 3L]
['oblong' 1L]
['obstacle' 1L]
['occasional' 1L]
['occasionally' 1L]
['occurred' 2L]
['odd' 1L]
['of' 499L]
['off' 62L]
['offend' 1L]
['offended' 10L]
['offer' 2L]
['officer' 1L]
['officers' 4L]
['often' 5L]
['oh' 10L]
['ointment' 1L]
['old' 17L]
['older' 2L]
['oldest' 1L]
['on' 189L]
['once' 31L]
['one' 93L]
["one's" 1L]
['ones' 1L]
['oneself' 1L]
['onions' 1L]
['only' 47L]
['oop' 7L]
['ootiful' 4L]
['open' 7L]
['opened' 10L]
['opening' 3L]
['opinion' 1L]
['opportunity' 8L]
['opposite' 1L]
['or' 76L]
['order' 3L]
['ordered' 3L]
["ordered'" 1L]
['ordering' 2L]
['ornamented' 2L]
['other' 40L]
['others' 7L]
['otherwise' 4L]
['ought' 14L]
['our' 7L]
['ourselves' 1L]
['out' 116L]
['outside' 3L]
['over' 40L]
['overcome' 1L]
['overhead' 1L]
['own' 10L]
['oyster' 1L]
['p' 1L]
['pace' 1L]
['pack' 5L]
['paint' 1L]
['painting' 2L]
['pair' 5L]
['pairs' 1L]
['pale' 4L]
['pan' 1L]
['panted' 1L]
['panting' 2L]
['paper' 4L]
['parchment' 2L]
['pardon' 6L]
['pardoned' 1L]
['part' 2L]
['particular' 4L]
['partner' 1L]
['partners' 1L]
['parts' 1L]
['party' 9L]
['pass' 1L]
['passage' 4L]
['passed' 5L]
['passing' 1L]
['passion' 3L]
['passionate' 1L]
['past' 3L]
['patience' 1L]
['patiently' 2L]
['patriotic' 1L]
['patted' 1L]
['pattering' 3L]
['pattern' 1L]
['pause' 2L]
['paused' 1L]
['paw' 3L]
['paws' 4L]
['pebbles' 2L]
['peeped' 3L]
['peeping' 1L]
['peering' 1L]
['pegs' 1L]
['pence' 1L]
['pencil' 2L]
['pencils' 1L]
['people' 13L]
['pepper' 6L]
['perfectly' 4L]
['perhaps' 14L]
['permitted' 1L]
['persisted' 2L]
['person' 4L]
['personal' 2L]
['pet' 1L]
['picked' 3L]
['picking' 2L]
['picture' 1L]
['pictured' 1L]
['pictures' 4L]
['pie' 3L]
['piece' 6L]
['pieces' 3L]
['pig' 9L]
['pigeon' 1L]
['pigs' 6L]
['pinch' 1L]
['pinched' 2L]
['pine' 1L]
['pink' 1L]
['piteous' 1L]
['pitied' 1L]
['pity' 3L]
['place' 8L]
['placed' 1L]
['places' 2L]
['plainly' 1L]
['plan' 4L]
['planning' 1L]
['plate' 3L]
['plates' 2L]
['play' 8L]
['played' 1L]
['players' 4L]
['playing' 2L]
['pleaded' 3L]
['pleasant' 1L]
['pleasanter' 1L]
['please' 13L]
['pleased' 7L]
['pleases' 1L]
['pleasing' 1L]
['pleasure' 2L]
['plenty' 1L]
['pocked' 1L]
['pocket' 5L]
['pointed' 1L]
['pointing' 4L]
['poison' 3L]
['poker' 1L]
['poky' 1L]
['politely' 6L]
['pool' 10L]
['poor' 25L]
['pop' 1L]
['pope' 1L]
['porpoise' 4L]
['position' 2L]
['positively' 1L]
['possible' 1L]
['possibly' 3L]
['pot' 1L]
['pounds' 1L]
['pour' 1L]
['poured' 1L]
['powdered' 1L]
['practice' 1L]
['present' 3L]
['presented' 1L]
['presents' 2L]
['pressed' 3L]
['pressing' 1L]
['pretend' 1L]
['pretending' 1L]
['pretexts' 1L]
['prettier' 1L]
['pretty' 1L]
['prevent' 1L]
['printed' 1L]
['prison' 1L]
['prisoner' 1L]
["prisoner's" 1L]
['prize' 1L]
['prizes' 3L]
['proceed' 2L]
['procession' 5L]
['processions' 1L]
['produced' 1L]
['producing' 1L]
['promise' 1L]
['promised' 1L]
['promising' 1L]
['pronounced' 1L]
['proper' 3L]
['proposal' 1L]
['prosecute' 1L]
['protection' 1L]
['proud' 2L]
['prove' 1L]
['proved' 2L]
['proves' 1L]
['provoking' 1L]
['puffed' 1L]
['pulled' 1L]
['pulling' 1L]
['pun' 1L]
['punching' 1L]
['punished' 1L]
['puppy' 6L]
["puppy's" 1L]
['purple' 1L]
['purpose' 1L]
['purring' 2L]
['push' 1L]
['put' 31L]
['putting' 3L]
['puzzle' 1L]
['puzzled' 9L]
['puzzling' 4L]
['quarrel' 1L]
['quarrelled' 1L]
['quarrelling' 2L]
['queer' 12L]
['queerest' 1L]
['question' 17L]
['questions' 4L]
['quick' 1L]
['quicker' 1L]
['quickly' 2L]
['quiet' 2L]
['quietly' 5L]
['quite' 53L]
['quiver' 1L]
['rabbit' 5L]
['rabbits' 1L]
['race' 5L]
['railway' 2L]
['raised' 2L]
['raising' 1L]
['ran' 16L]
['rapidly' 2L]
['rapped' 1L]
['rat' 1L]
['rate' 9L]
['rather' 25L]
['rats' 1L]
['rattle' 1L]
['rattling' 2L]
['raven' 1L]
['ravens' 1L]
['raving' 2L]
['raw' 1L]
['reach' 4L]
['reaching' 1L]
['read' 10L]
['readily' 1L]
['reading' 3L]
['ready' 8L]
['real' 3L]
['reality' 1L]
['really' 11L]
['rearing' 1L]
['reason' 9L]
['reasonable' 1L]
['reasons' 1L]
['received' 1L]
['recognised' 1L]
['recovered' 2L]
['red' 2L]
['reduced' 1L]
['reeds' 1L]
['refreshments' 1L]
['refused' 1L]
['regular' 2L]
['relief' 2L]
['relieved' 1L]
['remain' 1L]
['remained' 3L]
['remaining' 1L]
['remark' 10L]
['remarkable' 2L]
['remarked' 10L]
['remarking' 3L]
['remarks' 3L]
['remedies' 1L]
['remember' 14L]
['remembered' 5L]
['remembering' 1L]
['reminding' 1L]
['removed' 2L]
['repeat' 6L]
['repeated' 10L]
['repeating' 3L]
['replied' 29L]
['reply' 5L]
['resource' 1L]
['respect' 1L]
['respectable' 1L]
['respectful' 1L]
['rest' 10L]
['resting' 2L]
['result' 1L]
['retire' 1L]
['returned' 1L]
['returning' 1L]
['rich' 1L]
['riddle' 1L]
['riddles' 2L]
['ridge' 1L]
['ridges' 1L]
['ridiculous' 1L]
['right' 30L]
['righthand' 1L]
['rightly' 1L]
['ring' 2L]
['ringlets' 2L]
['riper' 1L]
['rippling' 1L]
['rise' 1L]
['rises' 1L]
['rising' 1L]
['roared' 1L]
['roast' 1L]
['rock' 1L]
['rocket' 1L]
['roof' 6L]
['room' 13L]
['roots' 2L]
['rope' 1L]
['rose' 4L]
['roses' 3L]
['rosetree' 1L]
['roughly' 1L]
['round' 41L]
['row' 2L]
['royal' 2L]
['rubbed' 1L]
['rubbing' 2L]
['rude' 2L]
['rudeness' 1L]
['rule' 4L]
['rules' 3L]
['rumbling' 1L]
['run' 3L]
['running' 8L]
['rush' 2L]
['rushed' 1L]
['rustled' 1L]
['rustling' 1L]
['sad' 3L]
['sadly' 5L]
['safe' 2L]
['sage' 1L]
['said' 456L]
['salmon' 1L]
['salt' 2L]
['same' 23L]
['sand' 1L]
['sands' 1L]
['sang' 2L]
['sat' 17L]
['saucepan' 1L]
['saucepans' 1L]
['saucer' 1L]
['savage' 4L]
['save' 1L]
['saves' 1L]
['saw' 13L]
['say' 52L]
['saying' 15L]
['says' 4L]
['scale' 1L]
['scaly' 1L]
['school' 6L]
['schoolroom' 1L]
['scolded' 1L]
['scrambling' 1L]
['scratching' 1L]
['scream' 2L]
['screamed' 4L]
['screaming' 1L]
['scroll' 2L]
['sea' 14L]
['search' 1L]
['seaside' 1L]
['seated' 1L]
['second' 4L]
['secondly' 2L]
['secret' 1L]
['see' 66L]
['seeing' 1L]
['seem' 8L]
['seemed' 27L]
['seems' 5L]
['seen' 15L]
['seldom' 1L]
['sell' 2L]
['send' 1L]
['sending' 2L]
['sensation' 2L]
['sense' 3L]
['sent' 2L]
['sentence' 5L]
['sentenced' 1L]
['series' 1L]
['seriously' 1L]
['serpent' 6L]
['serpents' 3L]
['set' 14L]
['setting' 1L]
['settle' 1L]
['settled' 3L]
['settling' 1L]
['seven' 1L]
['several' 4L]
['severely' 4L]
['severity' 1L]
['sh' 1L]
['shade' 1L]
['shake' 1L]
['shaking' 3L]
['shall' 23L]
["shan't" 5L]
['shape' 1L]
['shaped' 3L]
['share' 1L]
['shared' 1L]
['sharing' 1L]
['sharks' 1L]
['sharp' 6L]
['sharply' 4L]
['she' 501L]
["she'd" 1L]
["she'll" 2L]
["she's" 4L]
['shedding' 1L]
['sheep' 1L]
['shelves' 2L]
['shepherd' 1L]
['shifting' 1L]
['shilling' 1L]
['shillings' 1L]
['shingle' 1L]
['shining' 1L]
['shiny' 1L]
['shiver' 1L]
['shock' 1L]
['shoes' 6L]
['shook' 9L]
['shore' 4L]
['short' 4L]
['shorter' 2L]
['should' 27L]
['shoulder' 4L]
['shoulders' 4L]
["shouldn't" 5L]
['shouted' 9L]
['shouting' 2L]
['show' 3L]
['shower' 2L]
['showing' 2L]
['shriek' 5L]
['shrieked' 1L]
['shrieks' 1L]
['shrill' 5L]
['shrimp' 1L]
['shrink' 1L]
['shrinking' 4L]
['shut' 5L]
['shutting' 2L]
['shyly' 1L]
['side' 17L]
['sides' 4L]
['sigh' 4L]
['sighed' 5L]
['sighing' 3L]
['sight' 10L]
['sign' 1L]
['signed' 2L]
['signifies' 1L]
['signify' 1L]
['silence' 11L]
['silent' 7L]
['simple' 5L]
['simpleton' 1L]
['simply' 3L]
['since' 4L]
['sing' 5L]
['singers' 2L]
['singing' 2L]
['sink' 1L]
['sir' 5L]
["sir'" 1L]
['sister' 8L]
["sister's" 1L]
['sisters' 2L]
['sit' 6L]
['sits' 1L]
['sitting' 10L]
['six' 2L]
['sixpence' 1L]
['size' 13L]
['sizes' 1L]
['skimming' 1L]
['skirt' 1L]
['skurried' 1L]
['sky' 5L]
['slate' 4L]
['slates' 7L]
["slates'll" 1L]
['sleep' 6L]
['sleepy' 5L]
['slightest' 1L]
['slipped' 3L]
['slippery' 1L]
['slowly' 8L]
['small' 10L]
['smaller' 3L]
['smallest' 2L]
['smile' 2L]
['smiled' 2L]
['smiling' 2L]
['smoke' 1L]
['smoking' 2L]
['snail' 3L]
['snappishly' 1L]
['snatch' 2L]
['sneeze' 2L]
['sneezed' 1L]
['sneezes' 2L]
['sneezing' 6L]
['snorting' 1L]
['snout' 1L]
['so' 124L]
['sob' 1L]
['sobbed' 1L]
['sobbing' 3L]
['sobs' 4L]
['soft' 1L]
['softly' 1L]
['soldier' 1L]
['soldiers' 10L]
['solemn' 3L]
['solemnly' 4L]
['solid' 1L]
['some' 48L]
['somebody' 5L]
['somehow' 1L]
['someone' 1L]
['somersault' 2L]
['something' 17L]
['sometimes' 5L]
['somewhere' 2L]
['son' 1L]
['song' 7L]
['soon' 24L]
['sooner' 2L]
['soothing' 1L]
['sorrow' 2L]
['sorrowful' 2L]
['sorrows' 1L]
['sorry' 1L]
['sort' 20L]
['sorts' 3L]
['sound' 4L]
['sounded' 5L]
['sounds' 3L]
['soup' 4L]
['sour' 1L]
['spades' 1L]
['speak' 13L]
['speaker' 1L]
['speaking' 5L]
['spectacles' 3L]
['speech' 3L]
['speed' 1L]
['spell' 1L]
['spirited' 1L]
['spite' 1L]
['splash' 1L]
['splashed' 1L]
['splashing' 2L]
['splendidly' 1L]
['spoke' 17L]
['spoken' 1L]
['spoon' 2L]
['spot' 1L]
['sprawling' 1L]
['spread' 3L]
['spreading' 1L]
['squeaked' 1L]
['squeaking' 2L]
['squeeze' 1L]
['squeezed' 1L]
['stairs' 3L]
['stalk' 1L]
['stamping' 2L]
['stand' 5L]
['standing' 1L]
['star' 1L]
['staring' 3L]
['started' 2L]
['startled' 2L]
['state' 1L]
['station' 1L]
['stay' 5L]
['stays' 1L]
['steady' 1L]
['steam' 1L]
['sternly' 1L]
['stick' 4L]
['sticks' 1L]
['stiff' 1L]
['still' 12L]
['stingy' 1L]
['stirring' 2L]
['stockings' 1L]
['stole' 1L]
['stood' 7L]
['stool' 1L]
['stoop' 2L]
['stop' 5L]
['stopped' 3L]
['stopping' 1L]
['story' 7L]
['straight' 2L]
['straightened' 1L]
['straightening' 1L]
['strange' 5L]
['strength' 1L]
['stretched' 2L]
['stretching' 1L]
['strings' 1L]
['struck' 2L]
['stuff' 3L]
['stupid' 5L]
['stupidest' 1L]
['stupidly' 1L]
['subdued' 1L]
['subject' 6L]
['subjects' 1L]
['submitted' 1L]
['succeeded' 3L]
['such' 40L]
['sudden' 5L]
['suddenly' 12L]
['suet' 1L]
['sugar' 2L]
['suit' 3L]
['sulkily' 2L]
['sulky' 3L]
['summer' 2L]
['sun' 2L]
['supple' 1L]
['suppose' 12L]
['suppressed' 4L]
['sure' 20L]
['surprise' 5L]
['surprised' 7L]
['swallow' 1L]
['swallowed' 1L]
['swallowing' 1L]
['swam' 5L]
['sweet' 1L]
['swim' 3L]
['swimming' 2L]
['table' 17L]
['tail' 9L]
['tails' 3L]
['take' 18L]
['taken' 4L]
['takes' 2L]
['taking' 5L]
['tale' 3L]
['tales' 1L]
['talk' 14L]
['talking' 16L]
['taller' 2L]
['tart' 1L]
['tarts' 6L]
['taste' 2L]
['tasted' 3L]
['tastes' 1L]
['taught' 4L]
['tea' 18L]
['teaching' 1L]
['teacup' 3L]
['teacups' 2L]
['teapot' 1L]
['tears' 10L]
['teases' 1L]
['teeth' 1L]
['telescope' 3L]
['telescopes' 1L]
['tell' 27L]
['telling' 2L]
['tells' 2L]
['temper' 5L]
['tempered' 2L]
['ten' 5L]
['terms' 1L]
['terribly' 1L]
['terrier' 1L]
['terror' 1L]
['than' 23L]
['thank' 1L]
['thanked' 1L]
['that' 260L]
["that's" 15L]
['thatched' 1L]
['the' 1525L]
['their' 50L]
['theirs' 1L]
['them' 88L]
['themselves' 3L]
['then' 72L]
['there' 49L]
["there's" 16L]
['these' 11L]
['they' 111L]
["they'd" 4L]
["they'll" 4L]
["they're" 10L]
["they've" 1L]
['thick' 1L]
['thimble' 3L]
["thimble'" 1L]
['thin' 1L]
['thing' 49L]
['things' 31L]
['think' 50L]
['thinking' 10L]
['thirteen' 1L]
['this' 113L]
['thistle' 2L]
['thoroughly' 2L]
['those' 9L]
['though' 12L]
['thought' 74L]
['thoughtfully' 4L]
['thoughts' 2L]
['thousand' 2L]
['three' 28L]
['threw' 2L]
['throat' 2L]
['throne' 1L]
['through' 11L]
['throw' 3L]
['throwing' 2L]
['thrown' 1L]
['thump' 2L]
['thunder' 1L]
['thunderstorm' 1L]
['thy' 1L]
['tide' 1L]
['tidy' 1L]
['tie' 1L]
['tied' 1L]
['tight' 1L]
['till' 21L]
['time' 68L]
['times' 6L]
['timid' 3L]
['timidly' 9L]
['tinkling' 1L]
['tiny' 4L]
['tipped' 1L]
['tiptoe' 2L]
['tired' 7L]
['tittered' 1L]
['to' 725L]
['toast' 1L]
['today' 1L]
['toes' 3L]
['toffee' 1L]
['together' 9L]
['told' 6L]
['tomorrow' 1L]
['tone' 40L]
['tones' 2L]
['tongue' 4L]
['too' 25L]
['took' 23L]
['top' 7L]
['tops' 1L]
['toss' 1L]
['tossing' 3L]
['touch' 1L]
['tougher' 1L]
['towards' 1L]
['toys' 1L]
['trampled' 1L]
['traps' 1L]
['tray' 1L]
['treacle' 5L]
['treading' 2L]
['treat' 1L]
['treated' 1L]
['tree' 8L]
['trees' 7L]
['tremble' 1L]
['trembled' 2L]
['trembling' 6L]
['tremulous' 1L]
['trial' 7L]
["trial's" 3L]
['trials' 1L]
['trickling' 1L]
['tricks' 1L]
['tried' 19L]
['triumphantly' 2L]
['trot' 1L]
['trotting' 2L]
['trouble' 6L]
['true' 3L]
['trumpet' 3L]
['trusts' 1L]
['truth' 1L]
['truthful' 1L]
['try' 12L]
['trying' 14L]
['tucked' 3L]
['tulip' 1L]
['tumbled' 1L]
['tumbling' 2L]
['tunnel' 1L]
['tureen' 1L]
['turkey' 1L]
['turn' 8L]
['turned' 16L]
['turning' 12L]
['turns' 3L]
['turtles' 2L]
['tut' 1L]
['twelfth' 1L]
['twelve' 4L]
['twentieth' 1L]
['twenty' 2L]
['twice' 5L]
['twinkle' 5L]
['twinkled' 1L]
['twinkling' 4L]
['twist' 2L]
['two' 32L]
['uglify' 1L]
['uglifying' 1L]
['ugly' 2L]
['unable' 1L]
['uncivil' 1L]
['uncomfortable' 4L]
['uncomfortably' 1L]
['uncommon' 1L]
['uncommonly' 1L]
['uncorked' 1L]
['under' 16L]
['underneath' 1L]
['understand' 6L]
['understood' 1L]
['undertone' 2L]
['undo' 1L]
['undoing' 1L]
['uneasily' 2L]
['uneasy' 1L]
['unfolded' 2L]
['unfortunate' 3L]
['unhappy' 2L]
['unimportant' 3L]
['unjust' 1L]
['unless' 2L]
['unlocking' 1L]
['unpleasant' 2L]
['unrolled' 2L]
['until' 5L]
['untwist' 1L]
['unusually' 1L]
['unwillingly' 1L]
['up' 98L]
['upon' 26L]
['upright' 1L]
['upset' 3L]
['upsetting' 1L]
['upstairs' 1L]
['us' 14L]
['use' 18L]
['used' 13L]
['useful' 2L]
['using' 2L]
['usual' 5L]
['usually' 2L]
['usurpation' 1L]
['vague' 1L]
['vanished' 4L]
['vanishing' 1L]
['variations' 1L]
['various' 1L]
['vegetable' 1L]
['velvet' 1L]
['venture' 3L]
['ventured' 4L]
['verdict' 4L]
['verse' 4L]
['verses' 4L]
['very' 126L]
['vinegar' 1L]
['violence' 1L]
['violent' 2L]
['violently' 4L]
['voice' 47L]
['voices' 2L]
['vote' 1L]
['vulgar' 1L]
['wag' 1L]
['wags' 1L]
['waist' 1L]
['waistcoat' 1L]
['wait' 1L]
['waited' 11L]
['waiting' 8L]
['walk' 5L]
['walked' 10L]
['walking' 5L]
['walrus' 1L]
['wander' 1L]
['wandered' 2L]
['wandering' 2L]
['want' 9L]
['wanted' 4L]
['wants' 2L]
['warning' 1L]
['was' 352L]
['wash' 2L]
['washing' 2L]
["wasn't" 11L]
['waste' 1L]
['wasting' 2L]
['watch' 7L]
['watched' 2L]
['watching' 3L]
['water' 5L]
['waters' 1L]
['waving' 5L]
['way' 56L]
['ways' 1L]
['we' 19L]
["we're" 2L]
["we've" 2L]
['weak' 2L]
['wearily' 1L]
['week' 3L]
['weeks' 1L]
['welcome' 1L]
['well' 40L]
['went' 83L]
['wept' 1L]
['were' 84L]
["were'" 1L]
["weren't" 1L]
['wet' 2L]
['what' 90L]
["what's" 3L]
['whatever' 2L]
['when' 69L]
['whenever' 1L]
['where' 9L]
["where's" 1L]
['whereupon' 1L]
['wherever' 2L]
['whether' 11L]
['which' 41L]
['while' 20L]
['whiles' 1L]
['whiskers' 3L]
['whisper' 3L]
['whispered' 5L]
['whispers' 1L]
['whistle' 1L]
['whistling' 1L]
['white' 8L]
['whiting' 8L]
['who' 48L]
['whole' 13L]
['whom' 1L]
['whose' 2L]
['why' 13L]
['wide' 2L]
['wider' 1L]
['wife' 1L]
['wig' 2L]
['wild' 2L]
['wildly' 2L]
['will' 24L]
['win' 1L]
['wind' 2L]
['window' 8L]
['wine' 2L]
['wings' 1L]
['wink' 2L]
['winter' 1L]
['wise' 2L]
['wish' 21L]
['with' 175L]
['within' 2L]
['without' 26L]
['witness' 10L]
['wits' 1L]
['woke' 1L]
['woman' 2L]
['won' 2L]
["won't" 23L]
["won't'" 1L]
['wonder' 18L]
['wondered' 1L]
['wonderful' 2L]
['wondering' 7L]
['wood' 8L]
['wooden' 1L]
['word' 10L]
['words' 21L]
['wore' 1L]
['work' 8L]
['works' 1L]
['world' 7L]
['worm' 1L]
['worried' 1L]
['worry' 1L]
['worse' 3L]
['worth' 4L]
['would' 70L]
["wouldn't" 12L]
['wow' 4L]
['wrapping' 1L]
['wretched' 2L]
['wriggling' 1L]
['write' 5L]
['writing' 6L]
['written' 6L]
['wrong' 5L]
['wrote' 3L]
['yard' 1L]
['yards' 1L]
['yawned' 2L]
['yawning' 2L]
['ye' 1L]
['year' 2L]
['years' 1L]
['yelp' 1L]
['yer' 4L]
['yes' 1L]
['yesterday' 3L]
['yet' 21L]
['you' 312L]
["you'd" 8L]
["you'll" 4L]
["you're" 15L]
["you've" 6L]
['young' 5L]
['your' 53L]
['yours' 2L]
['yourself' 10L]
['youth' 6L]
['zigzag' 1L]

MapReduce using MRJOB

Find documentation for MRJOB at https://pythonhosted.org/mrjob/ A framework that allows you to do mapreduce jobs without HADOOP but will run the same jobs in an hadoop environment. ( DUMBO and Pydoop give you lower level access to HADOOP)

Before using it for the first time install this package using : pip install mrjob

If that does not work use the alternatives in provided on https://pythonhosted.org/mrjob/guides/quickstart.html#installation

%%writefile myfile.py

write/save cell contents into myfile.py (use -a to append). Another alias: %%file myfile.py %run myfile.py

run myfile.py and output results in the current cell %load myfile.py

load "import" myfile.py into the current cell


In [11]:
! pip install mrjob


Collecting mrjob
  Downloading https://files.pythonhosted.org/packages/95/24/b55a4d4799f49c4c4413233a9437a855c384b8c40daea5e8dec8ec1733f0/mrjob-0.6.5-py2.py3-none-any.whl (308kB)
Collecting google-cloud-dataproc>=0.2.0 (from mrjob)
  Downloading https://files.pythonhosted.org/packages/07/23/7ae8951f5bc68d1471b492a6b8ee938d3ec9d47ebb82dc8ea1e45899ec8e/google_cloud_dataproc-0.2.0-py2.py3-none-any.whl (131kB)
Requirement already satisfied: PyYAML>=3.08 in c:\users\ps\anaconda2\lib\site-packages (from mrjob) (3.12)
Collecting google-cloud-logging>=1.5.0 (from mrjob)
  Downloading https://files.pythonhosted.org/packages/93/a5/220c51fa50520991b7de756730fcc7bf719a29f5b4339eb9854d56ffdaf3/google_cloud_logging-1.7.0-py2.py3-none-any.whl (100kB)
Collecting botocore>=1.6.0 (from mrjob)
  Downloading https://files.pythonhosted.org/packages/22/fa/af1ad7aebe166932fe1cac5eae5413d090eca4d723829756c31fc53c174d/botocore-1.12.14-py2.py3-none-any.whl (4.7MB)
Collecting google-cloud-storage>=1.9.0 (from mrjob)
  Downloading https://files.pythonhosted.org/packages/3b/cd/4f58416260817ed968b9875d97bd2f6974fe8fdce7b724b2cdd70107b815/google_cloud_storage-1.12.0-py2.py3-none-any.whl (54kB)
Collecting boto3>=1.4.6 (from mrjob)
  Downloading https://files.pythonhosted.org/packages/8e/49/4616f3beaa9974aa275933218fe64b200f4177ff5f0753a3b72c5cec5dda/boto3-1.9.14-py2.py3-none-any.whl (128kB)
Collecting google-api-core[grpc]<2.0.0dev,>=0.1.0 (from google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/a5/be/de30100034c391f4c56e2543f1507eb1b30b3030bd9a6764dd6cfe7a954e/google_api_core-1.4.1-py2.py3-none-any.whl (53kB)
Collecting google-cloud-core<0.29dev,>=0.28.0 (from google-cloud-logging>=1.5.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/0f/41/ae2418b4003a14cf21c1c46d61d1b044bf02cf0f8f91598af572b9216515/google_cloud_core-0.28.1-py2.py3-none-any.whl
Requirement already satisfied: docutils>=0.10 in c:\users\ps\anaconda2\lib\site-packages (from botocore>=1.6.0->mrjob) (0.14)
Collecting jmespath<1.0.0,>=0.7.1 (from botocore>=1.6.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in c:\users\ps\anaconda2\lib\site-packages (from botocore>=1.6.0->mrjob) (2.7.3)
Requirement already satisfied: urllib3<1.24,>=1.20 in c:\users\ps\anaconda2\lib\site-packages (from botocore>=1.6.0->mrjob) (1.22)
Collecting google-resumable-media>=0.3.1 (from google-cloud-storage>=1.9.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/77/95/2e4020a54366423ddba715f89fb7ca456c8f048b15cada6cd6a54cf10e8c/google_resumable_media-0.3.1-py2.py3-none-any.whl
Collecting s3transfer<0.2.0,>=0.1.10 (from boto3>=1.4.6->mrjob)
  Downloading https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)
Collecting protobuf>=3.4.0 (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/77/78/a7f1ce761e2c738e209857175cd4f90a8562d1bde32868a8cd5290d58926/protobuf-3.6.1-py2.py3-none-any.whl (390kB)
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/58/cb/96dbb4e50e7a9d856e89cc9c8e36ab1055f9774f7d85f37e2156c1d79d9f/google_auth-1.5.1-py2.py3-none-any.whl (65kB)
Requirement already satisfied: six>=1.10.0 in c:\users\ps\anaconda2\lib\site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (1.11.0)
Requirement already satisfied: futures>=3.2.0; python_version < "3.2" in c:\users\ps\anaconda2\lib\site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (3.2.0)
Collecting googleapis-common-protos<2.0dev,>=1.5.3 (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/00/03/d25bed04ec8d930bcfa488ba81a2ecbf7eb36ae3ffd7e8f5be0d036a89c9/googleapis-common-protos-1.5.3.tar.gz
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in c:\users\ps\anaconda2\lib\site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (2.18.4)
Requirement already satisfied: pytz in c:\users\ps\anaconda2\lib\site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (2018.4)
Requirement already satisfied: setuptools>=34.0.0 in c:\users\ps\anaconda2\lib\site-packages (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (39.1.0)
Collecting grpcio>=1.8.2; extra == "grpc" (from google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/12/d7/b7311f393c785dd877259e6427a3abbf7c75871b53da19c70a59ecf4266a/grpcio-1.15.0-cp27-cp27m-win_amd64.whl (1.5MB)
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/0a/58/cbee863250b31d80f47401d04f34038db6766f95dea1cc909ea099c7e571/cachetools-2.1.0-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from google-auth<2.0.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/19/02/fa63f7ba30a0d7b925ca29d034510fc1ffde53264b71b4155022ddf3ab5d/pyasn1_modules-0.2.2-py2.py3-none-any.whl (62kB)
Collecting rsa>=3.1.4 (from google-auth<2.0.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\users\ps\anaconda2\lib\site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in c:\users\ps\anaconda2\lib\site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (2.6)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\ps\anaconda2\lib\site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (2018.4.16)
Requirement already satisfied: enum34>=1.0.4 in c:\users\ps\anaconda2\lib\site-packages (from grpcio>=1.8.2; extra == "grpc"->google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob) (1.1.6)
Collecting pyasn1<0.5.0,>=0.4.1 (from pyasn1-modules>=0.2.1->google-auth<2.0.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=0.1.0->google-cloud-dataproc>=0.2.0->mrjob)
  Downloading https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)
Building wheels for collected packages: googleapis-common-protos
  Running setup.py bdist_wheel for googleapis-common-protos: started
  Running setup.py bdist_wheel for googleapis-common-protos: finished with status 'done'
  Stored in directory: C:\Users\PS\AppData\Local\pip\Cache\wheels\62\45\af\649bbf07b6595fda010be1bda667cd56d0444d07afc6f8b687
Successfully built googleapis-common-protos
Installing collected packages: protobuf, cachetools, pyasn1, pyasn1-modules, rsa, google-auth, googleapis-common-protos, grpcio, google-api-core, google-cloud-dataproc, google-cloud-core, google-cloud-logging, jmespath, botocore, google-resumable-media, google-cloud-storage, s3transfer, boto3, mrjob
Successfully installed boto3-1.9.14 botocore-1.12.14 cachetools-2.1.0 google-api-core-1.4.1 google-auth-1.5.1 google-cloud-core-0.28.1 google-cloud-dataproc-0.2.0 google-cloud-logging-1.7.0 google-cloud-storage-1.12.0 google-resumable-media-0.3.1 googleapis-common-protos-1.5.3 grpcio-1.15.0 jmespath-0.9.3 mrjob-0.6.5 protobuf-3.6.1 pyasn1-0.4.4 pyasn1-modules-0.2.2 rsa-4.0 s3transfer-0.1.13
distributed 1.21.8 requires msgpack, which is not installed.
grin 1.2.1 requires argparse>=1.1, which is not installed.
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

In [ ]:
# %load code/MRWordFrequencyCount.py
# Do not run this cell it is just displaying the content of the code
from mrjob.job import MRJob

class MRWordFrequencyCount(MRJob):
    
    def mapper(self, _ , line):
        yield "chars", len(line)
        yield "words", len(line.split())
        yield "lines", 1
        
    def reducer(self, key, values):
        yield key, sum(values)
        
if __name__ == '__main__':
    MRWordFrequencyCount.run()

In [20]:
%run -G code/MRWordFrequencyCount.py  ./.mrjob.conf data/canterbury/alice29.txt


No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
INFO:mrjob.conf:No configs found; falling back on auto-configuration
No configs specified for inline runner
No configs specified for inline runner
No configs specified for inline runner
No configs specified for inline runner
No configs specified for inline runner
WARNING:mrjob.conf:No configs specified for inline runner
Running step 1 of 1...
Running step 1 of 1...
Running step 1 of 1...
Running step 1 of 1...
Running step 1 of 1...
INFO:mrjob.sim:Running step 1 of 1...
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000
INFO:mrjob.runner:Creating temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000
job output is in c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output
INFO:mrjob.runner:job output is in c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output...
INFO:mrjob.runner:Streaming final output from c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000\output...
"chars"	148687
"lines"	12
"words"	26475
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000...
INFO:mrjob.runner:Removing temp directory c:\users\ps\appdata\local\temp\MRWordFrequencyCount.PS.20181001.022038.947000...
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
ERROR:mrjob.runner:[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFrequencyCount.PS.20181001.022038.947000\\step\\000\\cache\\MRWordFrequencyCount.py'

MRJOB Word count function


In [23]:
# %load code\MRWordFreqCount.py
# Do not run this block of code just a display of the stored program
from mrjob.job import MRJob
import re

WORD_RE = re.compile(r"[\w']+")


class MRWordFreqCount(MRJob):

    def mapper(self, _, line):
        for word in WORD_RE.findall(line):
            yield word.lower(), 1

    def combiner(self, word, counts):
        yield word, sum(counts)

    def reducer(self, word, counts):
        yield word, sum(counts)


if __name__ == '__main__':
    MRWordFreqCount.run()


usage: usage: ipykernel_launcher.py [options] [input files]
ipykernel_launcher.py: error: unrecognized arguments: -f
An exception has occurred, use %tb to see the full traceback.

SystemExit: 2

In [ ]:


In [24]:
%run code/MRWordFreqCount.py data/canterbury/alice29.txt


No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
No configs found; falling back on auto-configuration
INFO:mrjob.conf:No configs found; falling back on auto-configuration
No configs specified for inline runner
No configs specified for inline runner
No configs specified for inline runner
No configs specified for inline runner
No configs specified for inline runner
No configs specified for inline runner
WARNING:mrjob.conf:No configs specified for inline runner
Running step 1 of 1...
Running step 1 of 1...
Running step 1 of 1...
Running step 1 of 1...
Running step 1 of 1...
Running step 1 of 1...
INFO:mrjob.sim:Running step 1 of 1...
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000
Creating temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000
INFO:mrjob.runner:Creating temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000
job output is in c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output
job output is in c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output
INFO:mrjob.runner:job output is in c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output...
Streaming final output from c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output...
INFO:mrjob.runner:Streaming final output from c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000\output...
"'"	1091
"'em"	3
"'tis"	5
"2"	1
"9"	1
"_i_"	2
"a"	632
"abide"	1
"able"	1
"about"	94
"above"	3
"absence"	1
"absurd"	2
"acceptance"	1
"accident"	2
"accidentally"	1
"account"	1
"accounting"	1
"accounts"	1
"accusation"	1
"accustomed"	1
"ache"	1
"across"	5
"act"	1
"actually"	1
"ada"	1
"added"	23
"adding"	1
"addressed"	2
"addressing"	1
"adjourn"	1
"adoption"	1
"advance"	3
"advantage"	3
"adventures"	7
"advice"	2
"advisable"	2
"advise"	1
"affair"	1
"affectionately"	1
"afford"	1
"afore"	1
"afraid"	12
"after"	43
"afterwards"	2
"again"	82
"against"	9
"age"	4
"ago"	2
"agony"	1
"agree"	2
"ah"	5
"ahem"	1
"air"	15
"airs"	1
"alarm"	2
"alarmed"	1
"alas"	4
"alice"	386
"alice's"	12
"alive"	3
"all"	182
"allow"	3
"almost"	6
"alone"	4
"along"	6
"aloud"	5
"already"	2
"also"	2
"altered"	1
"alternately"	1
"altogether"	5
"always"	13
"am"	15
"ambition"	1
"among"	12
"an"	57
"ancient"	1
"and"	872
"anger"	2
"angrily"	9
"angry"	5
"animal"	1
"animal's"	1
"animals"	4
"ann"	4
"annoy"	1
"annoyed"	1
"another"	22
"answer"	9
"answered"	4
"answers"	1
"antipathies"	1
"anxious"	3
"anxiously"	14
"any"	39
"anything"	20
"anywhere"	1
"appealed"	1
"appear"	2
"appearance"	1
"appeared"	8
"appearing"	1
"applause"	1
"apple"	1
"apples"	2
"arch"	1
"archbishop"	2
"arches"	4
"are"	54
"argue"	1
"argued"	1
"argument"	4
"arguments"	1
"arithmetic"	1
"arm"	15
"arms"	6
"around"	3
"arranged"	1
"arrived"	1
"arrow"	1
"arrum"	1
"as"	263
"ashamed"	2
"ask"	11
"askance"	1
"asked"	17
"asking"	5
"asleep"	8
"assembled"	2
"at"	211
"ate"	1
"atheling"	1
"atom"	2
"attempt"	1
"attempted"	1
"attempts"	1
"attended"	1
"attending"	3
"attends"	1
"audibly"	1
"australia"	1
"authority"	2
"avoid"	1
"away"	25
"awfully"	1
"axes"	1
"axis"	1
"baby"	14
"back"	39
"backs"	1
"bad"	2
"bag"	1
"baked"	1
"balanced"	1
"balls"	1
"bank"	3
"banks"	1
"banquet"	1
"bark"	2
"barking"	1
"barley"	1
"barrowful"	2
"bat"	3
"bathing"	1
"bats"	4
"bawled"	1
"be"	148
"beak"	1
"bear"	2
"beast"	1
"beasts"	2
"beat"	4
"beating"	2
"beau"	4
"beauti"	1
"beautiful"	13
"beautifully"	2
"beautify"	1
"became"	2
"because"	15
"become"	5
"becoming"	1
"bed"	1
"beds"	2
"bee"	1
"been"	38
"before"	38
"beg"	8
"began"	58
"begged"	1
"begin"	13
"beginning"	14
"begins"	4
"begun"	7
"behead"	1
"beheaded"	3
"beheading"	1
"behind"	13
"being"	19
"believe"	9
"believed"	1
"bells"	1
"belong"	1
"belongs"	2
"beloved"	1
"below"	3
"belt"	1
"bend"	2
"bent"	1
"besides"	4
"best"	12
"better"	13
"better'"	1
"between"	6
"bill"	13
"bill's"	4
"bird"	2
"birds"	10
"birthday"	1
"bit"	16
"bite"	2
"bitter"	1
"blacking"	1
"blades"	1
"blame"	1
"blasts"	2
"bleeds"	1
"blew"	2
"blow"	2
"blown"	1
"blows"	1
"body"	2
"boldly"	1
"bone"	1
"bones"	1
"book"	11
"books"	2
"boon"	1
"boots"	4
"bore"	1
"both"	14
"bother"	1
"bottle"	10
"bottom"	4
"bough"	1
"bound"	1
"bowed"	4
"bowing"	1
"box"	10
"boxed"	1
"boy"	3
"brain"	1
"branch"	1
"branches"	2
"brandy"	1
"brass"	1
"brave"	1
"bread"	7
"break"	2
"breath"	4
"breathe"	3
"breeze"	1
"bright"	8
"brightened"	2
"bring"	3
"bringing"	3
"bristling"	1
"broke"	2
"broken"	6
"brother's"	1
"brought"	3
"brown"	2
"brush"	1
"brushing"	1
"burn"	2
"burning"	1
"burnt"	1
"burst"	1
"bursting"	1
"busily"	4
"business"	7
"busy"	2
"but"	170
"butter"	9
"buttercup"	1
"buttered"	1
"butterfly"	1
"buttons"	1
"by"	58
"bye"	2
"c"	1
"cackled"	1
"cake"	3
"cakes"	3
"call"	9
"called"	15
"calling"	1
"calmly"	1
"came"	40
"camomile"	1
"can"	35
"can't"	28
"canary"	1
"candle"	3
"cannot"	1
"canterbury"	1
"canvas"	1
"capering"	1
"capital"	4
"cardboard"	1
"cards"	3
"care"	4
"carefully"	3
"cares"	2
"carried"	4
"carrier"	1
"carroll"	1
"carry"	1
"carrying"	2
"cart"	1
"cartwheels"	1
"case"	5
"cat"	35
"cat's"	2
"catch"	4
"catching"	2
"caterpillar"	27
"caterpillar's"	1
"cats"	13
"cattle"	1
"caucus"	3
"caught"	3
"cauldron"	2
"cause"	3
"caused"	2
"cautiously"	3
"ceiling"	1
"centre"	1
"certain"	3
"certainly"	14
"chain"	1
"chains"	1
"chair"	1
"chance"	4
"chanced"	1
"change"	13
"changed"	8
"changes"	2
"changing"	2
"chapter"	12
"character"	1
"charges"	1
"chatte"	1
"cheap"	1
"cheated"	1
"checked"	3
"cheeks"	1
"cheered"	3
"cheerfully"	1
"cherry"	1
"cheshire"	7
"chief"	1
"child"	11
"childhood"	1
"children"	10
"chimney"	6
"chimneys"	1
"chin"	7
"choice"	2
"choke"	1
"choked"	3
"choking"	1
"choosing"	1
"chop"	1
"chorus"	6
"chose"	2
"christmas"	1
"chrysalis"	1
"chuckled"	1
"circle"	1
"circumstances"	1
"civil"	3
"clamour"	1
"clapping"	1
"clasped"	1
"classics"	1
"claws"	2
"clean"	1
"clear"	2
"cleared"	1
"clearer"	1
"clearly"	1
"clever"	2
"climb"	1
"clinging"	1
"clock"	2
"close"	13
"closed"	2
"closely"	1
"closer"	1
"clubs"	1
"coast"	1
"coaxing"	2
"coils"	1
"cold"	1
"collar"	1
"collected"	2
"come"	46
"comes"	2
"comfits"	2
"comfort"	1
"comfortable"	1
"comfortably"	1
"coming"	9
"common"	1
"commotion"	1
"company"	1
"complained"	1
"complaining"	1
"completely"	1
"concert"	2
"concluded"	2
"conclusion"	2
"condemn"	1
"conduct"	1
"confused"	4
"confusing"	3
"confusion"	5
"conger"	1
"conqueror"	2
"conquest"	1
"consented"	1
"consider"	4
"considered"	3
"considering"	3
"constant"	2
"consultation"	1
"contempt"	1
"contemptuous"	1
"contemptuously"	2
"content"	1
"continued"	9
"contradicted"	1
"conversation"	10
"conversations"	1
"cook"	13
"cool"	2
"corner"	4
"corners"	1
"cost"	1
"could"	77
"couldn't"	9
"counting"	1
"country"	1
"couple"	1
"couples"	1
"courage"	3
"course"	26
"court"	18
"courtiers"	2
"coward"	1
"crab"	3
"crash"	3
"crashed"	1
"crawled"	1
"crawling"	1
"crazy"	1
"creature"	4
"creatures"	10
"creep"	1
"crept"	1
"cried"	20
"cries"	1
"crimson"	2
"crocodile"	1
"croquet"	9
"croqueted"	1
"croqueting"	1
"cross"	3
"crossed"	3
"crossly"	1
"crouched"	1
"crowd"	4
"crowded"	5
"crown"	3
"crumbs"	4
"crust"	1
"cry"	3
"crying"	2
"cucumber"	2
"cunning"	1
"cup"	2
"cupboards"	2
"cur"	1
"curiosity"	5
"curious"	19
"curiouser"	2
"curled"	2
"curls"	1
"curly"	1
"currants"	1
"curtain"	1
"curtsey"	1
"curtseying"	1
"curving"	1
"cushion"	2
"custard"	1
"custody"	2
"cut"	5
"cutting"	1
"d"	1
"dainties"	1
"daisies"	1
"daisy"	1
"dance"	13
"dancing"	2
"dare"	5
"daresay"	1
"dark"	3
"darkness"	1
"dates"	1
"daughter"	1
"day"	29
"days"	4
"dead"	4
"deal"	12
"dear"	29
"dears"	3
"death"	1
"decided"	3
"decidedly"	4
"declare"	2
"declared"	1
"deep"	7
"deepest"	1
"deeply"	4
"delay"	1
"delight"	3
"delighted"	2
"delightful"	2
"denial"	1
"denied"	2
"denies"	1
"deny"	2
"denying"	1
"depends"	1
"derision"	1
"deserved"	1
"desk"	1
"desks"	1
"despair"	1
"desperate"	1
"desperately"	1
"diamonds"	1
"did"	63
"didn't"	14
"die"	1
"died"	1
"different"	9
"difficult"	2
"difficulties"	1
"difficulty"	4
"dig"	1
"digging"	4
"diligently"	1
"dinah"	11
"dinah'll"	2
"dinah's"	1
"dinn"	2
"dinner"	2
"dipped"	2
"directed"	2
"direction"	5
"directions"	3
"directly"	2
"disagree"	1
"disappeared"	2
"disappointment"	1
"disgust"	1
"dish"	4
"dishes"	2
"dismay"	1
"disobey"	1
"dispute"	2
"distance"	8
"distant"	2
"distraction"	1
"dive"	1
"do"	81
"dodged"	1
"dodo"	13
"does"	9
"doesn't"	16
"dog"	2
"dog's"	1
"dogs"	3
"doing"	5
"don't"	61
"done"	15
"door"	30
"doors"	2
"doorway"	1
"dormouse"	39
"dormouse's"	1
"doth"	3
"double"	1
"doubled"	1
"doubling"	1
"doubt"	4
"doubtful"	2
"doubtfully"	2
"down"	102
"downward"	1
"downwards"	1
"doze"	1
"dozing"	1
"draggled"	1
"draw"	7
"drawing"	1
"drawling"	3
"dreadful"	2
"dreadfully"	6
"dream"	7
"dreamed"	1
"dreaming"	1
"dreamy"	1
"dressed"	1
"drew"	5
"dried"	1
"driest"	1
"drink"	7
"drinking"	1
"dripping"	1
"drive"	2
"drop"	1
"dropped"	5
"dropping"	1
"drowned"	1
"drunk"	2
"dry"	8
"duchess"	38
"duchess's"	3
"duck"	4
"dull"	3
"dunce"	1
"dutchess"	1
"e"	6
"each"	8
"eager"	3
"eagerly"	8
"eaglet"	3
"ear"	6
"earls"	2
"earnestly"	2
"ears"	5
"earth"	4
"easily"	3
"easy"	2
"eat"	18
"eaten"	1
"eating"	1
"eats"	1
"edgar"	1
"edge"	3
"edition"	1
"editions"	2
"educations"	1
"edwin"	2
"eel"	2
"eels"	1
"effect"	3
"egg"	1
"eggs"	5
"eh"	1
"either"	10
"elbow"	3
"elbows"	1
"elegant"	1
"eleventh"	1
"else"	11
"else's"	1
"elsie"	1
"emphasis"	1
"empty"	1
"encourage"	1
"encouraged"	1
"encouraging"	2
"end"	18
"ending"	2
"energetic"	1
"engaged"	1
"engine"	1
"england"	1
"english"	6
"engraved"	1
"enjoy"	1
"ennyworth"	1
"enormous"	1
"enough"	18
"entangled"	2
"entirely"	2
"entrance"	1
"escape"	4
"esq"	1
"est"	1
"even"	19
"evening"	5
"ever"	20
"every"	13
"everybody"	8
"everything"	12
"everything's"	2
"evidence"	7
"evidently"	1
"exact"	1
"exactly"	8
"examine"	2
"examining"	1
"excellent"	2
"except"	4
"exclaimed"	6
"exclamation"	1
"execute"	1
"executed"	6
"executes"	1
"execution"	3
"executioner"	5
"executioner's"	1
"executions"	2
"existence"	1
"expected"	1
"expecting"	3
"experiment"	2
"explain"	10
"explained"	1
"explanation"	2
"explanations"	1
"expressing"	1
"expression"	1
"extra"	1
"extraordinary"	2
"extras"	1
"extremely"	2
"eye"	7
"eyed"	1
"eyelids"	1
"eyes"	29
"face"	15
"faces"	5
"fact"	8
"fading"	1
"failure"	1
"faint"	1
"fainting"	1
"faintly"	1
"fair"	1
"fairly"	1
"fairy"	1
"fall"	7
"fallen"	4
"falling"	2
"familiarly"	1
"family"	1
"fan"	10
"fancied"	2
"fancy"	7
"fancying"	1
"fanned"	1
"fanning"	1
"far"	13
"farm"	1
"farmer"	1
"farther"	1
"fashion"	2
"fast"	4
"faster"	3
"fat"	1
"father"	6
"favoured"	1
"favourite"	1
"fear"	4
"feared"	1
"feather"	1
"feathers"	1
"feeble"	2
"feebly"	1
"feel"	8
"feeling"	7
"feelings"	2
"feet"	19
"fell"	6
"fellow"	4
"fellows"	1
"felt"	23
"fender"	1
"ferrets"	2
"fetch"	7
"few"	9
"fidgeted"	1
"field"	1
"fifteen"	1
"fifteenth"	1
"fifth"	1
"fig"	1
"fight"	2
"fighting"	1
"figure"	3
"figures"	1
"filled"	3
"fills"	1
"find"	21
"finding"	3
"finds"	1
"fine"	2
"finger"	5
"finish"	5
"finished"	12
"finishing"	1
"fire"	4
"fireplace"	1
"first"	51
"fish"	8
"fishes"	1
"fit"	3
"fits"	1
"fitted"	1
"five"	8
"fix"	1
"fixed"	1
"flame"	1
"flamingo"	5
"flamingoes"	2
"flapper"	1
"flappers"	1
"flashed"	1
"flat"	2
"flavour"	1
"flew"	1
"flinging"	1
"flock"	1
"floor"	3
"flower"	2
"flowers"	2
"flown"	1
"flung"	1
"flurry"	1
"flustered"	1
"fluttered"	1
"fly"	3
"flying"	1
"folded"	3
"folding"	1
"follow"	2
"followed"	8
"follows"	3
"fond"	4
"foolish"	1
"foot"	10
"footman"	13
"footman's"	1
"footmen"	1
"footsteps"	2
"for"	153
"forehead"	2
"forepaws"	1
"forget"	2
"forgetting"	3
"forgot"	2
"forgotten"	6
"fork"	1
"form"	1
"fortunately"	1
"forty"	1
"forwards"	1
"found"	32
"fountains"	2
"four"	8
"fourteenth"	1
"fourth"	1
"frame"	1
"frames"	1
"france"	1
"free"	3
"french"	4
"friend"	3
"friends"	2
"fright"	2
"frighten"	1
"frightened"	7
"frog"	3
"from"	36
"front"	2
"frontispiece"	1
"frowning"	4
"frying"	1
"ful"	1
"fulcrum"	1
"full"	6
"fumbled"	1
"fun"	3
"funny"	3
"fur"	3
"furious"	1
"furiously"	1
"furrow"	1
"furrows"	1
"further"	3
"fury"	3
"gained"	1
"gallons"	1
"game"	12
"game's"	1
"games"	1
"garden"	16
"gardeners"	8
"gather"	1
"gave"	15
"gay"	1
"gazing"	1
"general"	3
"generally"	7
"gently"	3
"geography"	1
"get"	46
"getting"	22
"giddy"	2
"girl"	4
"girls"	3
"give"	12
"given"	1
"giving"	2
"glad"	11
"glanced"	1
"glaring"	1
"glass"	10
"globe"	1
"gloomily"	1
"gloves"	11
"go"	50
"goes"	7
"going"	27
"golden"	7
"goldfish"	2
"gone"	13
"good"	27
"goose"	2
"got"	45
"graceful"	1
"grammar"	1
"grand"	3
"grant"	1
"grass"	4
"grave"	3
"gravely"	3
"gravy"	1
"grazed"	1
"great"	39
"green"	4
"grew"	1
"grey"	1
"grief"	1
"grin"	6
"grinned"	3
"grinning"	1
"grins"	1
"ground"	8
"grow"	13
"growing"	11
"growl"	3
"growled"	1
"growling"	1
"growls"	1
"grown"	7
"grumbled"	1
"grunt"	1
"grunted"	4
"gryphon"	54
"guard"	1
"guess"	3
"guessed"	3
"guests"	3
"guilt"	1
"guinea"	6
"had"	178
"hadn't"	8
"hair"	7
"half"	23
"hall"	9
"hand"	21
"handed"	3
"hands"	12
"handsome"	1
"handwriting"	1
"hanging"	3
"happen"	8
"happened"	7
"happening"	1
"happens"	5
"happy"	1
"hard"	8
"hardly"	12
"hare"	31
"harm"	1
"has"	7
"hasn't"	2
"haste"	1
"hastily"	16
"hat"	1
"hatching"	1
"hate"	2
"hated"	1
"hatter"	55
"hatter's"	1
"hatters"	1
"have"	80
"haven't"	8
"having"	10
"he"	122
"he'd"	1
"he'll"	1
"he's"	3
"head"	49
"head's"	1
"heads"	10
"heap"	1
"hear"	14
"heard"	30
"hearing"	4
"heart"	2
"hearth"	1
"hearthrug"	1
"hearts"	8
"heavy"	2
"hedge"	2
"hedgehog"	7
"hedgehogs"	3
"hedges"	1
"heels"	1
"height"	5
"held"	4
"help"	9
"helped"	1
"helpless"	1
"her"	247
"herald"	1
"here"	51
"hers"	4
"herself"	83
"hid"	1
"hide"	1
"high"	16
"highest"	1
"him"	43
"himself"	6
"hint"	2
"hippopotamus"	1
"his"	96
"hiss"	1
"histories"	1
"history"	7
"hit"	2
"hjckrrh"	1
"hm"	1
"hoarse"	3
"hoarsely"	1
"hold"	10
"holding"	3
"hole"	5
"holiday"	1
"hollow"	1
"home"	5
"honest"	1
"honour"	4
"hookah"	5
"hope"	3
"hoped"	1
"hopeful"	1
"hopeless"	1
"hoping"	3
"horse"	1
"hot"	7
"hour"	2
"hours"	4
"house"	18
"housemaid"	1
"houses"	1
"how"	68
"however"	20
"howled"	1
"howling"	3
"humble"	1
"humbly"	2
"hundred"	1
"hung"	1
"hungry"	3
"hunting"	3
"hurried"	11
"hurriedly"	2
"hurry"	11
"hurrying"	1
"hurt"	3
"hush"	3
"i"	408
"i'd"	11
"i'll"	31
"i'm"	59
"i've"	34
"idea"	15
"idiot"	1
"idiotic"	1
"if"	96
"ignorant"	1
"ii"	1
"iii"	1
"ill"	2
"imagine"	2
"imitated"	1
"immediate"	1
"immediately"	3
"immense"	1
"impatient"	1
"impatiently"	5
"impertinent"	1
"important"	7
"impossible"	3
"improve"	1
"in"	369
"incessantly"	1
"inches"	6
"inclined"	1
"indeed"	16
"indignant"	1
"indignantly"	4
"injure"	1
"ink"	1
"inkstand"	1
"inquired"	1
"inquisitively"	1
"inside"	2
"insolence"	1
"instance"	3
"instantly"	5
"instead"	3
"insult"	1
"interest"	1
"interesting"	5
"interrupt"	1
"interrupted"	9
"interrupting"	2
"into"	67
"introduce"	2
"introduced"	1
"invent"	1
"invented"	1
"invitation"	2
"invited"	2
"involved"	1
"inwards"	1
"irons"	1
"irritated"	1
"is"	108
"isn't"	7
"it"	530
"it'll"	8
"it's"	57
"its"	56
"itself"	14
"iv"	1
"ix"	1
"jack"	1
"jar"	2
"jaw"	1
"jaws"	2
"jelly"	1
"jogged"	1
"join"	9
"joined"	3
"journey"	1
"joys"	1
"judge"	4
"judging"	1
"jug"	1
"jumped"	6
"jumping"	4
"juror"	1
"jurors"	4
"jury"	22
"jurymen"	4
"just"	52
"justice"	1
"keep"	11
"keeping"	2
"kept"	13
"kettle"	1
"key"	9
"kick"	3
"kid"	5
"kill"	1
"killing"	1
"kills"	1
"kind"	7
"kindly"	2
"king"	61
"king's"	2
"kings"	1
"kiss"	1
"kissed"	1
"kitchen"	4
"knave"	9
"knee"	5
"kneel"	1
"knelt"	1
"knew"	14
"knife"	3
"knock"	1
"knocked"	1
"knocking"	3
"knot"	2
"know"	88
"knowing"	2
"knowledge"	3
"known"	1
"knows"	2
"knuckles"	1
"label"	2
"labelled"	1
"lacie"	1
"lad"	1
"ladder"	1
"lady"	3
"laid"	2
"lamps"	1
"land"	1
"languid"	1
"lap"	2
"large"	33
"larger"	7
"largest"	1
"lark"	1
"last"	33
"lasted"	2
"lastly"	1
"late"	6
"lately"	1
"later"	3
"latin"	1
"latitude"	2
"laugh"	1
"laughed"	2
"laughing"	2
"laughter"	1
"law"	2
"lay"	4
"lazily"	1
"lazy"	1
"leaders"	1
"leading"	1
"leaning"	2
"leant"	1
"leap"	1
"learn"	7
"learned"	1
"learning"	2
"learnt"	2
"least"	9
"leave"	9
"leaves"	6
"leaving"	1
"led"	4
"ledge"	1
"left"	14
"lefthand"	2
"legged"	2
"legs"	3
"length"	1
"less"	4
"lessen"	1
"lesson"	3
"lessons"	10
"lest"	1
"let"	17
"let's"	5
"letter"	3
"letters"	1
"lewis"	1
"licking"	1
"lie"	2
"life"	12
"lifted"	1
"like"	85
"liked"	6
"likely"	5
"likes"	1
"limbs"	1
"line"	2
"lines"	1
"linked"	1
"lips"	1
"list"	3
"listen"	7
"listened"	1
"listeners"	1
"listening"	3
"lit"	1
"little"	128
"live"	8
"lived"	3
"livery"	3
"lives"	4
"living"	2
"lizard"	5
"lizard's"	1
"lobster"	7
"lobsters"	6
"lock"	1
"locked"	1
"locks"	2
"lodging"	1
"london"	1
"lonely"	2
"long"	32
"longed"	2
"longer"	3
"longitude"	2
"look"	29
"looked"	45
"looking"	32
"loose"	1
"lory"	7
"lose"	1
"losing"	1
"lost"	3
"loud"	6
"louder"	1
"loudly"	3
"love"	3
"loveliest"	1
"lovely"	2
"loving"	1
"low"	15
"lower"	1
"lowing"	1
"luckily"	2
"lullaby"	1
"lying"	8
"m"	4
"ma"	2
"ma'am"	1
"mabel"	4
"machines"	1
"mad"	15
"made"	30
"magic"	1
"magpie"	1
"majesty"	12
"make"	27
"makes"	11
"making"	8
"mallets"	1
"man"	5
"manage"	7
"managed"	4
"managing"	1
"manner"	2
"manners"	1
"many"	12
"maps"	1
"march"	34
"marched"	1
"mark"	3
"marked"	6
"marmalade'"	1
"mary"	4
"master"	4
"matter"	9
"matters"	2
"may"	13
"maybe"	2
"mayn't"	1
"me"	66
"me'"	2
"meal"	1
"mean"	10
"meaning"	8
"means"	5
"meant"	5
"meanwhile"	1
"measure"	1
"meat"	1
"meekly"	2
"meet"	2
"meeting"	1
"melancholy"	6
"memorandum"	1
"memory"	1
"men'"	1
"mentioned"	3
"mercia"	2
"merely"	2
"merrily"	1
"messages"	2
"met"	3
"mice"	4
"middle"	7
"might"	28
"mile"	2
"miles"	3
"milk"	2
"millennium"	1
"mind"	11
"minded"	1
"minding"	1
"mine"	10
"mineral"	1
"minute"	21
"minutes"	11
"mischief"	1
"miserable"	2
"miss"	4
"missed"	2
"mistake"	3
"mixed"	2
"mock"	56
"moderate"	1
"modern"	1
"moment"	29
"moment's"	2
"month"	2
"moon"	1
"moral"	8
"morals"	1
"morcar"	2
"more"	49
"morning"	5
"morsel"	1
"most"	8
"mostly"	2
"mournful"	1
"mournfully"	1
"mouse"	43
"mouse's"	1
"mouth"	10
"mouths"	4
"move"	3
"moved"	5
"moving"	3
"much"	51
"muchness"	3
"muddle"	1
"multiplication"	1
"murder"	1
"murdering"	1
"muscular"	1
"mushroom"	8
"music"	3
"must"	44
"mustard"	3
"muttered"	2
"muttering"	3
"my"	58
"myself"	7
"mystery"	2
"name"	10
"names"	2
"narrow"	2
"nasty"	1
"natural"	4
"natured"	1
"naturedly"	1
"nay"	1
"near"	15
"nearer"	5
"nearly"	11
"neat"	1
"neatly"	2
"neck"	7
"needn't"	3
"needs"	1
"neighbour"	1
"neighbouring"	1
"neither"	2
"nervous"	5
"nest"	1
"never"	47
"never'"	1
"nevertheless"	1
"new"	5
"newspapers"	1
"next"	30
"nibbled"	2
"nibbling"	3
"nice"	6
"nicely"	2
"night"	5
"nile"	1
"nine"	5
"no"	90
"nobody"	8
"nodded"	1
"noise"	3
"noises"	1
"none"	4
"nonsense"	7
"nor"	3
"normans"	1
"northumbria"	2
"nose"	7
"nose'"	1
"not"	144
"not'"	1
"note"	2
"nothing"	34
"notice"	5
"noticed"	8
"noticing"	1
"notion"	3
"now"	60
"nowhere"	2
"number"	5
"nurse"	3
"nursing"	3
"o"	3
"o'clock"	3
"obliged"	3
"oblong"	1
"obstacle"	1
"occasional"	1
"occasionally"	1
"occurred"	2
"odd"	1
"of"	513
"off"	73
"offend"	1
"offended"	10
"offer"	2
"officer"	1
"officers"	4
"often"	5
"oh"	45
"ointment"	1
"old"	19
"older"	2
"oldest"	1
"on"	193
"once"	34
"one"	103
"one's"	1
"ones"	1
"oneself"	1
"onions"	1
"only"	50
"oop"	7
"ootiful"	4
"open"	7
"opened"	10
"opening"	3
"opinion"	1
"opportunity"	8
"opposite"	1
"or"	77
"orange"	1
"order"	3
"ordered"	3
"ordered'"	1
"ordering"	2
"ornamented"	2
"other"	40
"others"	7
"otherwise"	4
"ou"	1
"ought"	14
"our"	8
"ours"	1
"ourselves"	1
"out"	117
"outside"	4
"over"	40
"overcome"	1
"overhead"	1
"owl"	3
"own"	10
"oyster"	1
"p"	1
"pace"	1
"pack"	5
"paint"	1
"painting"	2
"pair"	5
"pairs"	1
"pale"	4
"pan"	1
"panted"	1
"panther"	3
"panting"	2
"paper"	4
"parchment"	2
"pardon"	6
"pardoned"	1
"paris"	2
"part"	2
"particular"	4
"partner"	1
"partners"	1
"parts"	1
"party"	10
"pass"	1
"passage"	4
"passed"	5
"passing"	1
"passion"	3
"passionate"	1
"past"	3
"pat"	3
"patience"	1
"patiently"	2
"patriotic"	1
"patted"	1
"pattering"	3
"pattern"	1
"pause"	2
"paused"	1
"paw"	3
"paws"	4
"pebbles"	2
"peeped"	3
"peeping"	1
"peering"	1
"pegs"	1
"pence"	1
"pencil"	2
"pencils"	1
"pennyworth"	1
"people"	13
"pepper"	8
"perfectly"	4
"perhaps"	17
"permitted"	1
"persisted"	2
"person"	4
"personal"	2
"persons"	1
"pet"	1
"picked"	3
"picking"	2
"picture"	1
"pictured"	1
"pictures"	4
"pie"	3
"piece"	6
"pieces"	3
"pig"	11
"pigeon"	12
"pigs"	6
"pinch"	2
"pinched"	2
"pine"	1
"pink"	1
"piteous"	1
"pitied"	1
"pity"	3
"place"	8
"placed"	1
"places"	2
"plainly"	1
"plan"	4
"planning"	1
"plate"	3
"plates"	2
"play"	8
"played"	1
"players"	4
"playing"	2
"pleaded"	3
"pleasant"	1
"pleasanter"	1
"please"	19
"pleased"	7
"pleases"	1
"pleasing"	1
"pleasure"	2
"plenty"	2
"pocked"	1
"pocket"	6
"pointed"	1
"pointing"	4
"poison"	3
"poker"	1
"poky"	1
"politely"	6
"pool"	11
"poor"	27
"pop"	1
"pope"	1
"porpoise"	4
"position"	2
"positively"	1
"possible"	1
"possibly"	3
"pot"	1
"pounds"	1
"pour"	1
"poured"	1
"powdered"	1
"practice"	1
"pray"	3
"precious"	1
"present"	3
"presented"	1
"presently"	2
"presents"	2
"pressed"	3
"pressing"	1
"pretend"	1
"pretending"	1
"pretexts"	1
"prettier"	1
"pretty"	1
"prevent"	1
"printed"	1
"prison"	1
"prisoner"	1
"prisoner's"	1
"prize"	1
"prizes"	5
"proceed"	2
"procession"	5
"processions"	1
"produced"	1
"producing"	1
"promise"	1
"promised"	1
"promising"	1
"pronounced"	1
"proper"	3
"proposal"	1
"prosecute"	1
"protection"	1
"proud"	2
"prove"	1
"proved"	2
"proves"	2
"provoking"	1
"puffed"	1
"pulled"	1
"pulling"	1
"pun"	1
"punching"	1
"punished"	1
"puppy"	6
"puppy's"	1
"purple"	1
"purpose"	1
"purring"	2
"push"	1
"puss"	1
"put"	31
"putting"	3
"puzzle"	1
"puzzled"	9
"puzzling"	4
"quadrille"	4
"quarrel"	1
"quarrelled"	1
"quarrelling"	2
"queen"	68
"queen's"	7
"queens"	1
"queer"	12
"queerest"	1
"question"	17
"questions"	4
"quick"	2
"quicker"	1
"quickly"	2
"quiet"	2
"quietly"	5
"quite"	55
"quiver"	1
"rabbit"	46
"rabbit'"	1
"rabbit's"	4
"rabbits"	1
"race"	6
"railway"	2
"raised"	2
"raising"	1
"ran"	16
"rapidly"	2
"rapped"	1
"rat"	1
"rate"	9
"rather"	25
"rats"	1
"rattle"	1
"rattling"	2
"raven"	1
"ravens"	1
"raving"	2
"raw"	1
"reach"	4
"reaching"	1
"read"	11
"readily"	1
"reading"	3
"ready"	8
"real"	3
"reality"	1
"really"	13
"rearing"	1
"reason"	9
"reasonable"	1
"reasons"	1
"received"	1
"recognised"	1
"recovered"	2
"red"	3
"reduced"	1
"reeds"	1
"reeling"	1
"refreshments"	1
"refused"	1
"regular"	2
"relief"	2
"relieved"	1
"remain"	1
"remained"	3
"remaining"	1
"remark"	10
"remarkable"	2
"remarked"	10
"remarking"	3
"remarks"	3
"remedies"	1
"remember"	14
"remembered"	5
"remembering"	1
"reminding"	1
"removed"	2
"repeat"	7
"repeated"	10
"repeating"	3
"replied"	29
"reply"	5
"resource"	1
"respect"	1
"respectable"	1
"respectful"	1
"rest"	10
"resting"	2
"result"	1
"retire"	1
"returned"	2
"returning"	1
"rich"	1
"riddle"	1
"riddles"	2
"ridge"	1
"ridges"	1
"ridiculous"	1
"right"	32
"righthand"	1
"rightly"	1
"ring"	2
"ringlets"	2
"riper"	1
"rippling"	1
"rise"	1
"rises"	1
"rising"	1
"roared"	1
"roast"	1
"rock"	1
"rocket"	1
"rome"	2
"roof"	6
"room"	13
"roots"	2
"rope"	1
"rose"	4
"roses"	3
"rosetree"	1
"roughly"	1
"round"	41
"row"	2
"royal"	2
"rubbed"	1
"rubbing"	2
"rude"	2
"rudeness"	1
"rule"	5
"rules"	3
"rumbling"	1
"run"	4
"running"	8
"rush"	2
"rushed"	1
"rustled"	1
"rustling"	1
"sad"	3
"sadly"	5
"safe"	2
"sage"	1
"said"	462
"salmon"	1
"salt"	2
"same"	24
"sand"	1
"sands"	1
"sang"	2
"sat"	17
"saucepan"	1
"saucepans"	1
"saucer"	1
"savage"	4
"save"	1
"saves"	1
"saw"	13
"say"	52
"saying"	15
"says"	4
"scale"	1
"scaly"	1
"school"	6
"schoolroom"	1
"scolded"	1
"scrambling"	1
"scratching"	1
"scream"	2
"screamed"	4
"screaming"	1
"scroll"	2
"sea"	14
"seals"	1
"seaography"	1
"search"	1
"seaside"	1
"seated"	1
"second"	4
"secondly"	2
"secret"	1
"see"	67
"seeing"	1
"seem"	8
"seemed"	27
"seems"	5
"seen"	15
"seldom"	1
"sell"	2
"send"	1
"sending"	2
"sends"	1
"sensation"	2
"sense"	3
"sent"	2
"sentence"	6
"sentenced"	1
"series"	1
"seriously"	1
"serpent"	9
"serpents"	3
"set"	14
"setting"	1
"settle"	1
"settled"	3
"settling"	1
"seven"	6
"several"	4
"severely"	4
"severity"	1
"sh"	2
"shade"	1
"shake"	1
"shakespeare"	1
"shaking"	3
"shall"	25
"shan't"	6
"shape"	1
"shaped"	3
"share"	1
"shared"	1
"sharing"	1
"shark"	1
"sharks"	1
"sharp"	6
"sharply"	4
"she"	540
"she'd"	2
"she'll"	3
"she's"	7
"shedding"	1
"sheep"	1
"shelves"	2
"shepherd"	1
"shifting"	1
"shilling"	1
"shillings"	1
"shingle"	1
"shining"	1
"shiny"	1
"shiver"	1
"shock"	1
"shoes"	7
"shook"	9
"shore"	4
"short"	4
"shorter"	2
"should"	27
"shoulder"	4
"shoulders"	4
"shouldn't"	5
"shouted"	9
"shouting"	2
"show"	3
"shower"	2
"showing"	2
"shriek"	5
"shrieked"	1
"shrieks"	1
"shrill"	5
"shrimp"	1
"shrink"	1
"shrinking"	4
"shut"	5
"shutting"	2
"shy"	1
"shyly"	1
"side"	17
"sides"	4
"sigh"	4
"sighed"	5
"sighing"	3
"sight"	10
"sign"	1
"signed"	2
"signifies"	1
"signify"	1
"silence"	14
"silent"	7
"simple"	5
"simpleton"	1
"simply"	3
"since"	4
"sing"	6
"singers"	2
"singing"	2
"sink"	1
"sir"	6
"sir'"	1
"sister"	8
"sister's"	1
"sisters"	2
"sit"	8
"sits"	1
"sitting"	10
"six"	2
"sixpence"	1
"sixteenth"	1
"size"	13
"sizes"	1
"skimming"	1
"skirt"	1
"skurried"	1
"sky"	5
"slate"	4
"slates"	7
"slates'll"	1
"sleep"	6
"sleepy"	5
"slightest"	1
"slipped"	3
"slippery"	1
"slowly"	8
"sluggard"	1
"small"	10
"smaller"	3
"smallest"	2
"smile"	2
"smiled"	2
"smiling"	2
"smoke"	1
"smoking"	2
"snail"	3
"snappishly"	1
"snatch"	2
"sneeze"	2
"sneezed"	1
"sneezes"	2
"sneezing"	6
"snorting"	1
"snout"	1
"so"	151
"sob"	1
"sobbed"	1
"sobbing"	3
"sobs"	4
"soft"	1
"softly"	1
"soldier"	1
"soldiers"	10
"solemn"	3
"solemnly"	4
"soles"	1
"solid"	1
"some"	51
"somebody"	7
"somehow"	1
"someone"	1
"somersault"	2
"something"	18
"sometimes"	5
"somewhere"	3
"son"	1
"song"	7
"soo"	7
"soon"	25
"sooner"	2
"soothing"	1
"sorrow"	2
"sorrowful"	2
"sorrows"	1
"sorry"	1
"sort"	20
"sorts"	3
"sound"	4
"sounded"	5
"sounds"	4
"soup"	18
"sour"	1
"spades"	1
"speak"	15
"speaker"	1
"speaking"	5
"spectacles"	3
"speech"	3
"speed"	1
"spell"	1
"spirited"	1
"spite"	1
"splash"	1
"splashed"	1
"splashing"	2
"splendidly"	1
"spoke"	17
"spoken"	1
"spoon"	2
"spot"	1
"sprawling"	1
"spread"	3
"spreading"	1
"squeaked"	1
"squeaking"	2
"squeeze"	1
"squeezed"	1
"stairs"	3
"stalk"	1
"stamping"	2
"stand"	6
"standing"	1
"star"	1
"staring"	3
"started"	2
"startled"	2
"state"	1
"station"	1
"stay"	5
"stays"	1
"steady"	1
"steam"	1
"sternly"	1
"stick"	4
"sticks"	1
"stiff"	1
"stigand"	1
"still"	13
"stingy"	1
"stirring"	2
"stockings"	1
"stole"	2
"stolen"	1
"stood"	7
"stool"	1
"stoop"	2
"stop"	6
"stopped"	3
"stopping"	1
"story"	8
"straight"	2
"straightened"	1
"straightening"	1
"strange"	5
"strength"	1
"stretched"	2
"stretching"	2
"strings"	1
"struck"	2
"stuff"	4
"stupid"	6
"stupidest"	1
"stupidly"	1
"subdued"	1
"subject"	6
"subjects"	1
"submitted"	1
"succeeded"	3
"such"	41
"sudden"	5
"suddenly"	13
"suet"	1
"sugar"	2
"suit"	3
"sulkily"	2
"sulky"	3
"summer"	2
"sun"	2
"supple"	1
"suppose"	14
"suppress"	1
"suppressed"	4
"sure"	24
"surprise"	5
"surprised"	7
"swallow"	1
"swallowed"	1
"swallowing"	1
"swam"	5
"sweet"	1
"swim"	5
"swimming"	2
"t"	1
"table"	18
"tail"	9
"tails"	3
"take"	22
"taken"	4
"takes"	2
"taking"	5
"tale"	4
"tales"	1
"talk"	14
"talking"	17
"taller"	2
"tart"	1
"tarts"	7
"taste"	2
"tasted"	3
"tastes"	1
"taught"	4
"tea"	19
"teaching"	1
"teacup"	3
"teacups"	2
"teapot"	1
"tears"	11
"teases"	1
"teeth"	1
"telescope"	3
"telescopes"	1
"tell"	32
"telling"	2
"tells"	2
"temper"	5
"tempered"	2
"ten"	6
"terms"	1
"terribly"	1
"terrier"	1
"terror"	1
"than"	24
"thank"	4
"thanked"	1
"that"	280
"that'll"	1
"that's"	34
"thatched"	1
"the"	1642
"their"	52
"theirs"	1
"them"	88
"themselves"	3
"then"	94
"there"	75
"there's"	24
"therefore"	1
"these"	14
"they"	131
"they'd"	4
"they'll"	4
"they're"	13
"they've"	1
"thick"	1
"thimble"	3
"thimble'"	1
"thin"	1
"thing"	49
"things"	31
"think"	53
"thinking"	11
"thirteen"	1
"this"	134
"thistle"	2
"thoroughly"	2
"those"	10
"though"	13
"thought"	74
"thoughtfully"	4
"thoughts"	2
"thousand"	2
"three"	28
"threw"	2
"throat"	2
"throne"	1
"through"	12
"throw"	3
"throwing"	2
"thrown"	1
"thump"	2
"thunder"	1
"thunderstorm"	1
"thy"	1
"tide"	1
"tidy"	1
"tie"	1
"tied"	1
"tight"	1
"till"	21
"tillie"	1
"time"	71
"times"	6
"timid"	3
"timidly"	9
"tinkling"	1
"tiny"	4
"tipped"	1
"tiptoe"	2
"tired"	7
"tittered"	1
"to"	729
"toast"	1
"today"	1
"toes"	3
"toffee"	1
"together"	9
"told"	6
"tomorrow"	1
"tone"	40
"tones"	2
"tongue"	4
"too"	26
"took"	24
"top"	7
"tops"	1
"tortoise"	3
"toss"	1
"tossing"	3
"touch"	1
"tougher"	1
"towards"	1
"toys"	1
"trampled"	1
"traps"	1
"tray"	1
"treacle"	7
"treading"	2
"treat"	1
"treated"	1
"tree"	8
"trees"	7
"tremble"	1
"trembled"	2
"trembling"	6
"tremulous"	1
"trial"	7
"trial's"	3
"trials"	1
"trickling"	1
"tricks"	1
"tried"	19
"trims"	1
"triumphantly"	2
"trot"	1
"trotting"	2
"trouble"	6
"true"	4
"trumpet"	3
"trusts"	1
"truth"	1
"truthful"	1
"try"	12
"trying"	14
"tucked"	3
"tulip"	1
"tumbled"	1
"tumbling"	2
"tunnel"	1
"tureen"	1
"turkey"	1
"turn"	11
"turned"	16
"turning"	12
"turns"	3
"turtle"	57
"turtle's"	2
"turtles"	2
"tut"	2
"twelfth"	1
"twelve"	4
"twentieth"	1
"twenty"	3
"twice"	5
"twinkle"	8
"twinkled"	1
"twinkling"	4
"twist"	2
"two"	40
"ugh"	2
"uglification"	2
"uglify"	1
"uglifying"	1
"ugly"	2
"unable"	1
"uncivil"	1
"uncomfortable"	4
"uncomfortably"	1
"uncommon"	1
"uncommonly"	1
"uncorked"	1
"under"	16
"underneath"	1
"understand"	6
"understood"	1
"undertone"	2
"undo"	1
"undoing"	1
"uneasily"	2
"uneasy"	1
"unfolded"	2
"unfortunate"	3
"unhappy"	2
"unimportant"	5
"unjust"	1
"unless"	2
"unlocking"	1
"unpleasant"	2
"unrolled"	2
"until"	5
"untwist"	1
"unusually"	1
"unwillingly"	1
"up"	100
"upon"	26
"upright"	1
"upset"	3
"upsetting"	1
"upstairs"	1
"us"	14
"use"	18
"used"	13
"useful"	2
"using"	2
"usual"	5
"usually"	2
"usurpation"	1
"v"	1
"vague"	1
"vanished"	4
"vanishing"	1
"variations"	1
"various"	1
"vegetable"	1
"velvet"	1
"venture"	3
"ventured"	4
"verdict"	4
"verse"	4
"verses"	4
"very"	144
"vi"	1
"vii"	1
"viii"	1
"vinegar"	1
"violence"	1
"violent"	2
"violently"	4
"visit"	1
"voice"	48
"voices"	2
"vote"	1
"vulgar"	1
"w"	1
"wag"	1
"wags"	1
"waist"	1
"waistcoat"	2
"wait"	1
"waited"	11
"waiting"	9
"wake"	2
"walk"	5
"walked"	10
"walking"	5
"walrus"	1
"wander"	1
"wandered"	2
"wandering"	2
"want"	9
"wanted"	4
"wants"	2
"warning"	1
"was"	357
"wash"	2
"washing"	3
"wasn't"	11
"waste"	1
"wasting"	2
"watch"	8
"watched"	2
"watching"	3
"water"	5
"waters"	1
"waving"	5
"way"	56
"ways"	1
"we"	30
"we're"	2
"we've"	2
"weak"	2
"wearily"	1
"week"	3
"weeks"	1
"welcome"	1
"well"	63
"went"	83
"wept"	1
"were"	84
"were'"	1
"weren't"	1
"wet"	2
"what"	136
"what's"	5
"whatever"	3
"when"	79
"whenever"	1
"where"	13
"where's"	2
"whereupon"	1
"wherever"	2
"whether"	11
"which"	49
"while"	25
"whiles"	1
"whiskers"	3
"whisper"	3
"whispered"	5
"whispers"	1
"whistle"	1
"whistling"	1
"white"	30
"whiting"	8
"who"	61
"who's"	2
"whoever"	1
"whole"	13
"whom"	1
"whose"	2
"why"	40
"wide"	2
"wider"	1
"wife"	1
"wig"	2
"wild"	2
"wildly"	2
"will"	33
"william"	7
"william's"	1
"win"	1
"wind"	2
"window"	8
"wine"	2
"wings"	1
"wink"	2
"winter"	1
"wise"	2
"wish"	21
"with"	180
"within"	2
"without"	26
"witness"	10
"wits"	1
"woke"	1
"woman"	2
"won"	2
"won't"	23
"won't'"	1
"wonder"	18
"wondered"	1
"wonderful"	2
"wondering"	7
"wonderland"	3
"wood"	8
"wooden"	1
"word"	10
"words"	21
"wore"	1
"work"	8
"works"	1
"world"	7
"worm"	1
"worried"	1
"worry"	1
"worse"	3
"worth"	4
"would"	83
"wouldn't"	13
"wow"	6
"wrapping"	1
"wretched"	2
"wriggling"	1
"write"	6
"writhing"	1
"writing"	6
"written"	6
"wrong"	5
"wrote"	3
"x"	1
"xi"	1
"xii"	1
"yard"	1
"yards"	1
"yawned"	2
"yawning"	2
"ye"	1
"year"	2
"years"	1
"yelp"	1
"yer"	4
"yes"	13
"yesterday"	3
"yet"	25
"you"	365
"you'd"	10
"you'll"	6
"you're"	23
"you've"	7
"young"	5
"your"	62
"yours"	3
"yourself"	10
"youth"	6
"zealand"	1
"zigzag"	1
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000...
Removing temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000...
INFO:mrjob.runner:Removing temp directory c:\users\ps\appdata\local\temp\MRWordFreqCount.PS.20181001.022200.276000...
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
ERROR:mrjob.runner:[Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'
Traceback (most recent call last):
  File "C:\Users\PS\anaconda2\lib\site-packages\mrjob\runner.py", line 615, in _cleanup_local_tmp
    shutil.rmtree(self._local_tmp_dir)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 261, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Users\PS\anaconda2\lib\shutil.py", line 264, in rmtree
    os.remove(fullname)
WindowsError: [Error 5] Access is denied: u'c:\\users\\ps\\appdata\\local\\temp\\MRWordFreqCount.PS.20181001.022200.276000\\step\\000\\cache\\MRWordFreqCount.py'

In [26]:
!pip install pyspark


Collecting pyspark
  Downloading https://files.pythonhosted.org/packages/5e/cb/d8ff49ba885e2c88b8cf2967edd84235ffa9ac301bffef657dfa5605a112/pyspark-2.3.2.tar.gz (211.9MB)
Collecting py4j==0.10.7 (from pyspark)
  Downloading https://files.pythonhosted.org/packages/e3/53/c737818eb9a7dc32a7cd4f1396e787bd94200c3997c72c1dbe028587bd76/py4j-0.10.7-py2.py3-none-any.whl (197kB)
Building wheels for collected packages: pyspark
  Running setup.py bdist_wheel for pyspark: started
  Running setup.py bdist_wheel for pyspark: still running...
  Running setup.py bdist_wheel for pyspark: finished with status 'done'
  Stored in directory: C:\Users\PS\AppData\Local\pip\Cache\wheels\be\7d\34\cd3cfbc75d8b6b6ae0658e5425348560b86d187fe3e53832cc
Successfully built pyspark
Installing collected packages: py4j, pyspark
Successfully installed py4j-0.10.7 pyspark-2.3.2
distributed 1.21.8 requires msgpack, which is not installed.
grin 1.2.1 requires argparse>=1.1, which is not installed.
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Another way to implement this is to use SPARK on your local box: Need to load spark and pyspark to get all this working

http://spark.apache.org/downloads.html

pyspark: pip install pyspark

On windows you need to follow: https://medium.com/@GalarnykMichael/install-spark-on-windows-pyspark-4498a5d8d66c

Test code From: https://github.com/eBay/Spark/blob/master/examples/src/main/python/wordcount.py


In [ ]:
# %load code\pysparkcount.py
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#https://github.com/apache/spark/blob/master/examples/src/main/python/wordcount.py

from __future__ import print_function

import sys
from operator import add

from pyspark import SparkContext


if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: wordcount <file>", file=sys.stderr)
        exit(-1)
    sc = SparkContext(appName="PythonWordCount")
    lines = sc.textFile(sys.argv[1], 1)
    counts = lines.flatMap(lambda x: x.split(' ')) \
                  .map(lambda x: (x, 1)) \
                  .reduceByKey(add)
    output = counts.collect()
    for (word, count) in output:
        print("%s: %i" % (word, count))

    sc.stop()

Should be able to run using %run code/pysparkcount.py data/canterbury/alice29.txt


In [ ]: